-- Logs begin at Thu 2022-07-21 21:42:46 UTC, end at Thu 2022-07-21 23:14:04 UTC. -- Jul 21 22:33:49.669815 dv-cinder-850697-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 21 22:33:49.678914 dv-cinder-850697-1-iscsi devstack@keystone.service[87154]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 21 22:33:49.680927 dv-cinder-850697-1-iscsi devstack@keystone.service[87154]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jul 21 22:33:49 2022] *** Jul 21 22:33:49.680927 dv-cinder-850697-1-iscsi devstack@keystone.service[87154]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 21 22:33:49.680927 dv-cinder-850697-1-iscsi devstack@keystone.service[87154]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 21 22:33:49.680927 dv-cinder-850697-1-iscsi devstack@keystone.service[87154]: nodename: dv-cinder-850697-1-iscsi Jul 21 22:33:49.680927 dv-cinder-850697-1-iscsi devstack@keystone.service[87154]: machine: x86_64 Jul 21 22:33:49.680927 dv-cinder-850697-1-iscsi devstack@keystone.service[87154]: clock source: unix Jul 21 22:33:49.680927 dv-cinder-850697-1-iscsi devstack@keystone.service[87154]: pcre jit disabled Jul 21 22:33:49.680927 dv-cinder-850697-1-iscsi devstack@keystone.service[87154]: detected number of CPU cores: 4 Jul 21 22:33:49.680927 dv-cinder-850697-1-iscsi devstack@keystone.service[87154]: current working directory: / Jul 21 22:33:49.680927 dv-cinder-850697-1-iscsi devstack@keystone.service[87154]: detected binary path: /usr/bin/uwsgi-core Jul 21 22:33:49.680927 dv-cinder-850697-1-iscsi devstack@keystone.service[87154]: your processes number limit is 47680 Jul 21 22:33:49.681524 dv-cinder-850697-1-iscsi devstack@keystone.service[87154]: your memory page size is 4096 bytes Jul 21 22:33:49.681524 dv-cinder-850697-1-iscsi devstack@keystone.service[87154]: detected max file descriptor number: 2048 Jul 21 22:33:49.681524 dv-cinder-850697-1-iscsi devstack@keystone.service[87154]: lock engine: pthread robust mutexes Jul 21 22:33:49.681524 dv-cinder-850697-1-iscsi devstack@keystone.service[87154]: thunder lock: enabled Jul 21 22:33:49.681524 dv-cinder-850697-1-iscsi devstack@keystone.service[87154]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 21 22:33:49.681524 dv-cinder-850697-1-iscsi devstack@keystone.service[87154]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 21 22:33:49.726973 dv-cinder-850697-1-iscsi devstack@keystone.service[87154]: Python main interpreter initialized at 0x55d571c6d9c0 Jul 21 22:33:49.726973 dv-cinder-850697-1-iscsi devstack@keystone.service[87154]: python threads support enabled Jul 21 22:33:49.726973 dv-cinder-850697-1-iscsi devstack@keystone.service[87154]: your server socket listen backlog is limited to 100 connections Jul 21 22:33:49.726973 dv-cinder-850697-1-iscsi devstack@keystone.service[87154]: your mercy for graceful operations on workers is 90 seconds Jul 21 22:33:49.727340 dv-cinder-850697-1-iscsi devstack@keystone.service[87154]: mapped 403077 bytes (393 KB) for 2 cores Jul 21 22:33:49.727418 dv-cinder-850697-1-iscsi devstack@keystone.service[87154]: *** Operational MODE: preforking *** Jul 21 22:33:49.727669 dv-cinder-850697-1-iscsi devstack@keystone.service[87154]: *** uWSGI is running in multiple interpreter mode *** Jul 21 22:33:49.727669 dv-cinder-850697-1-iscsi devstack@keystone.service[87154]: spawned uWSGI master process (pid: 87154) Jul 21 22:33:49.727877 dv-cinder-850697-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 21 22:33:49.730809 dv-cinder-850697-1-iscsi devstack@keystone.service[87154]: spawned uWSGI worker 1 (pid: 87167, cores: 1) Jul 21 22:33:49.730809 dv-cinder-850697-1-iscsi devstack@keystone.service[87154]: spawned uWSGI worker 2 (pid: 87168, cores: 1) Jul 21 22:33:49.730809 dv-cinder-850697-1-iscsi devstack@keystone.service[87154]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 21 22:33:55.181157 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 21 22:33:55.181157 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: from cryptography.utils import int_from_bytes Jul 21 22:33:55.181963 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 21 22:33:55.181963 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: from cryptography.utils import int_from_bytes Jul 21 22:33:55.182375 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 21 22:33:55.182375 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: from cryptography.utils import int_from_bytes Jul 21 22:33:55.182484 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 21 22:33:55.182484 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: from cryptography.utils import int_from_bytes Jul 21 22:33:55.230684 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 21 22:33:55.230911 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 21 22:33:55.231105 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] command line args: [] {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 21 22:33:55.231226 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 21 22:33:55.231373 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 21 22:33:55.231458 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 21 22:33:55.232075 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ================================================================================ {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 21 22:33:55.232155 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] command line args: [] {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 21 22:33:55.232329 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] admin_token = **** {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.232416 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 21 22:33:55.232572 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] config_dir = [] {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.232649 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ================================================================================ {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 21 22:33:55.232755 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.232840 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] admin_token = **** {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.232918 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] config_source = [] {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.233075 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] config_dir = [] {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.233153 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] debug = True {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.233284 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.233371 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.233629 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] config_source = [] {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.233629 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] debug = True {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.233866 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.233995 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.234135 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] insecure_debug = False {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.234258 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.234378 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.234493 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] insecure_debug = False {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.234612 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.234739 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.234897 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] list_limit = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.235020 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.235135 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] log_config_append = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.235255 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] list_limit = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.235370 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.235493 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] log_config_append = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.235614 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] log_dir = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.235728 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.235848 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] log_file = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.235962 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] log_dir = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.235962 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] log_file = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.236190 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.236313 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.236423 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.236525 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.236647 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.236758 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.236880 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.236880 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.237106 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.237223 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.237339 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.237339 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.237572 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.237693 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.237809 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.237930 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.238044 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.238164 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.238278 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.238399 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.238521 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.238635 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.238759 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.238902 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.239026 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.239154 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.239269 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] notification_format = cadf {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.239387 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] notification_format = cadf {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.239510 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.239623 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.239738 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.239859 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.240016 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] publish_errors = False {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.240132 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] publish_errors = False {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.240245 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.240365 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.240487 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.240609 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.240724 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.240838 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.240951 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.241084 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.241209 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.241345 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.241501 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] standard_threads = False {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.241625 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] standard_threads = False {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.241741 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] strict_password_check = False {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.241863 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] strict_password_check = False {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.241984 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.241984 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] use_eventlog = False {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.242212 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.242333 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] use_journal = False {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.242455 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] use_eventlog = False {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.242581 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] use_json = False {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.242698 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] use_journal = False {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.242852 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] use_stderr = False {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.242976 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] use_json = False {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.243091 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] use_syslog = False {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.243205 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] use_stderr = False {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.243205 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] use_syslog = False {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.243503 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] watch_log_file = False {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.243617 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] watch_log_file = False {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 22:33:55.243733 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.243853 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.243974 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.243974 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.244199 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.244313 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.244426 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.244539 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.244658 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.244782 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.244898 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.245075 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.245194 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.245308 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.245422 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.245546 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.245662 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.245778 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.245891 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.246005 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.246127 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.246243 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.246363 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.246483 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.246598 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.246726 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.246883 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.247007 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.247123 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.247839 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.247961 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.248082 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.248207 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.248324 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.248439 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.248560 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.248681 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.248797 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.248924 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.249045 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.249166 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.249288 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.249402 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.249521 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.249712 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.249837 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.249959 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.250080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.250080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.250302 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.250425 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.250546 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.250661 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.250789 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.250905 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.251029 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.251156 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.251279 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.251394 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.251508 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.251622 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.251735 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.251849 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.251969 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.252085 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.252198 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.252296 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.252403 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.252511 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.252615 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.252711 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.252815 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.252911 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.253005 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.253110 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.253212 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.253325 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.253430 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.253580 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.253700 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.253816 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.253931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.254045 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.254161 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.254275 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.254398 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.254514 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.254629 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.254803 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.254934 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.255050 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.255174 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.255300 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.255417 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.255540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.255667 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.255783 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.255905 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.256020 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.256143 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.256250 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.256364 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.256478 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.256592 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.256706 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.256830 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.256937 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.257050 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.257164 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.257278 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.257401 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.257517 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.257631 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.257746 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.257860 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.257975 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.258096 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.258219 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.258336 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.258452 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.258573 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.258688 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.258829 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.258954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.259079 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.259196 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.259316 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.259444 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.259569 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.259693 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.259808 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.259923 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.260038 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.260158 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.260274 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.260400 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.260517 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.260684 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.260808 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.260931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.261047 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.261239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.261358 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.261481 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.261597 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.261597 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.261814 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.261937 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] cache.enabled = True {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.262053 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.262175 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.262292 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.262415 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.262531 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.262664 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.262851 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] cache.enabled = True {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.262971 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.263088 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.263203 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.263318 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.263433 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.263557 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.263674 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.263790 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.263905 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.264026 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.264143 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.264266 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.264383 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.264498 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.264614 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.264729 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.264845 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.264968 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.265085 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.265200 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.265322 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.265439 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.265554 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.265679 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.265796 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.265911 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.266026 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.266141 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.266256 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.266380 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.266496 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.266611 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.266734 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.266861 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.266976 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.267103 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.267221 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.267337 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.267452 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.267573 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.267689 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.267814 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.267930 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.268046 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.268161 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.268266 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.268379 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.268502 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.268619 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.268734 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.268850 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] auth.external = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.268965 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.269079 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] auth.mapped = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.269205 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.269322 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.269442 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.269558 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.269673 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] auth.external = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.269819 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] auth.password = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.269950 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] auth.mapped = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.270070 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] auth.token = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.270070 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.270307 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.270433 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] catalog.caching = True {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.270550 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.270550 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] auth.password = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.270789 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.270924 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] auth.token = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.271043 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.271160 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.271276 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.271392 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] catalog.caching = True {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.271514 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.271514 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.271743 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.271859 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] credential.caching = True {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.271975 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.272091 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] credential.driver = sql {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.272216 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.272334 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.272450 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.272566 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.272681 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.272805 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.272930 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] credential.caching = True {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.273048 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.273165 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] credential.driver = sql {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.273281 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.273397 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.273513 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.273637 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.273761 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.273886 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.274003 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.274120 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.274236 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.274363 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.274480 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.274603 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.274719 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.274887 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.275003 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.275202 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.275323 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.275439 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.275555 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.275671 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.275787 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] federation.caching = True {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.275910 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.276027 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.276143 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.276260 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] federation.driver = sql {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.276375 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.276491 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.276615 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.276732 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.276848 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] federation.caching = True {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.276964 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.277080 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.277196 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.277321 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] federation.driver = sql {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.277439 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.277564 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.277673 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.277787 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.277902 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.278027 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.278145 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.278263 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.278523 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.278652 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.278785 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] identity.caching = True {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.278917 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.279065 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.279183 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.279301 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.279418 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.279535 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.279666 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.279791 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.279914 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] identity.caching = True {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.280031 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] identity.driver = sql {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.280148 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.280264 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.280393 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.280543 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.280660 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.280783 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.280901 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.281017 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.281142 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] identity.driver = sql {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.281259 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.281259 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.281479 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.281596 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.281712 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.281837 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.281954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.282071 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.282186 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.282309 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.282425 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.282551 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.282669 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.282815 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.282935 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.282935 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.283165 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.283284 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.283406 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.283406 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.283626 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.283742 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.283866 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.283984 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.284099 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.284216 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.284331 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.284447 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.284571 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.284688 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.284804 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.284920 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.285035 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.285150 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.285274 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.285392 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.285507 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.285624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.285748 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.285866 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.285989 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.286107 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.286223 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.286339 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.286454 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.286570 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.286696 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.286826 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.286944 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.287060 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.287175 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.287290 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ldap.password = **** {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.287418 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.287537 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.287653 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.287770 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.287887 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.288003 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.288127 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.288245 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.288361 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.288477 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.288592 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ldap.password = **** {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.288708 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.288834 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.288952 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.289068 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.289184 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.289299 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.289422 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.289551 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.289674 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.289791 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.289908 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.290024 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.290139 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.290139 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.290367 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.290483 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.290600 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.290715 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ldap.user = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.290891 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.291020 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.291139 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.291255 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.291372 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.291488 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.291604 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.291729 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.291846 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.291964 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.292086 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.292203 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.292319 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ldap.user = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.292443 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.292561 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.292677 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.292800 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.292918 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.293035 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.293160 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.293278 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.293394 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.293511 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.293627 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.293749 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.293875 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.294572 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.294692 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.294820 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.294939 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.295057 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.295187 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.295312 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.295428 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.295545 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.295660 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.295785 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.295903 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.296020 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.296138 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.296254 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.296370 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.296494 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.296613 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.296730 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.296847 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.296963 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.297079 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.297205 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.297323 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.297439 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.297556 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.297672 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.297789 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] policy.driver = sql {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.297914 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.298032 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.298148 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.298266 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.298382 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.298498 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.298688 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.298820 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] receipt.caching = True {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.298939 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.299055 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.299172 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.299299 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.299425 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] policy.driver = sql {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.299543 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.299664 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.299780 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.299780 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.299998 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.300123 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] resource.caching = True {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.300241 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.300358 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.300473 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] receipt.caching = True {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.300589 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] resource.driver = sql {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.300705 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.300829 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.300947 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.301063 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.301179 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.301294 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.301411 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.301535 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] revoke.caching = True {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.301653 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.301769 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.301885 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] resource.caching = True {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.302000 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.302116 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.302240 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] role.cache_time = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.302359 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] resource.driver = sql {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.302475 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] role.caching = True {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.302591 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.302707 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] role.driver = sql {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.302872 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.303002 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] role.list_limit = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.303121 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.303238 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.303354 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] revoke.caching = True {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.303472 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.303589 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.303713 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.303831 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.303947 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.304063 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] role.cache_time = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.304178 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.304295 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] role.caching = True {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.304423 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.304541 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] role.driver = sql {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.304658 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.304774 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] role.list_limit = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.304890 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.305005 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.305129 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.305247 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.305364 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.305480 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.305598 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.305714 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.305840 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.305958 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.306075 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.306192 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.306307 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.306433 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.306551 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.306667 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.306797 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.306916 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.307032 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.307159 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.307279 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.307395 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.307511 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.307628 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.307745 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.307869 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.307987 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.308103 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.308220 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.308335 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.308451 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.308575 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.308694 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.308694 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.308912 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.309029 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.309144 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.309272 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.309390 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.309507 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.309627 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.310344 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.310464 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.310590 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.310709 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.310853 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.310971 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.311087 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] token.cache_time = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.311203 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.311332 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] token.caching = True {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.311451 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.311574 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.311691 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.311808 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] token.provider = fernet {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.311924 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.312049 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.312166 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.312282 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.312398 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.312514 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.312630 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.312754 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.312872 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.312989 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.313105 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] token.cache_time = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.313221 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.313337 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] token.caching = True {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.313461 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] trust.driver = sql {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.313579 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.313695 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.313811 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] token.provider = fernet {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.313928 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.314045 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.314170 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.314288 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.314405 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.314521 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.314637 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.314762 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.314902 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.314902 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.315121 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.315121 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.315338 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.315466 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] trust.driver = sql {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.315585 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] database.connection = **** {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.315709 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.315835 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.315953 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.316070 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.316192 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.316309 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.316434 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.316550 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.316666 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.316782 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.316897 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.317013 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.317143 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.317261 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.317261 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.317480 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.317595 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.317711 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] database.connection = **** {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.317839 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.318035 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.318157 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.318274 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.318391 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.318508 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.318623 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.318748 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.318919 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.319037 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.319154 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.319270 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.319387 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.319515 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.319638 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.319756 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.319872 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.319872 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 21 22:33:55.320091 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.320091 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.320091 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.320091 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.320091 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.320091 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.320091 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.320091 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.320091 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.321037 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 22:33:55.321037 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 21 22:33:55.346212 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=87167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.346445 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=87167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.346671 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=87167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.347768 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=87168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.348009 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=87168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.348243 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=87168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.355244 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=87167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.357410 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=87168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.384160 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=87167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.384390 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=87167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.385664 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=87168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.385903 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=87168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.526953 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=87168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.531364 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=87167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.576646 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=87168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.576886 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=87168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.576995 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=87167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.577233 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=87167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.641498 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.641723 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=87167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.641946 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=87167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.642029 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.642162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=87167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.642247 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=87168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.642375 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=87167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.642457 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=87168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.642588 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=87167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.642680 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=87168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.642851 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=87168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.643075 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=87168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.668335 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.668570 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=87167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.668671 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.668799 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=87167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.668883 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=87168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.669599 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=87167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.669688 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=87168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.669833 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=87167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.669925 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=87168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.670052 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=87167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.670135 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=87168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.670265 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=87168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.670346 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.670483 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.681407 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=87167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.681812 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=87168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.694035 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.694264 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=87167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.694462 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.694547 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=87167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.694695 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=87167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.694806 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=87168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.694939 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=87167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.695033 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=87168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.695153 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=87167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.695237 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=87168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.695365 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.695446 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=87168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.695578 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=87168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.695794 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 22:33:55.712603 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.712811 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 21 22:33:55.713144 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.713345 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 21 22:33:55.713613 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.713873 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.713954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.714128 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 21 22:33:55.714209 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.714377 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.714467 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.714664 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 21 22:33:55.714967 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.715313 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.715576 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.715834 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.725906 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.726182 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.726449 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.726731 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.728175 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.728457 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.728729 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.729017 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.735724 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=87167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:33:55.738017 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=87168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:33:55.739031 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=87167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:33:55.739308 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.739587 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.739974 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.740257 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.740518 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.740787 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.741059 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.741324 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.741405 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=87168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:33:55.741593 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.741682 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.741859 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.741940 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.742281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.742572 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.742867 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.743146 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.743429 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.743706 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.743985 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.744260 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.768399 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.769961 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=87167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:33:55.770237 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.771051 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.772681 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=87168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:33:55.772964 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.775531 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=87167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:33:55.775809 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.776087 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.778309 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=87168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:33:55.779129 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.779414 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.784145 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=87167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:33:55.784419 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.786808 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=87168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:33:55.787077 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.789099 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=87167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:33:55.789383 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.789664 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.789946 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.790220 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.790490 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.790755 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.791043 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.791742 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=87168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:33:55.792032 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.792318 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.792603 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.792882 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.793157 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.793429 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.793698 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.810212 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.810483 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.813064 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.813343 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.815103 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=87167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:33:55.818050 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=87168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:33:55.818731 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.819044 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.821880 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.822181 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.824056 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=87167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:33:55.827824 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=87168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:33:55.828262 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=87167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:33:55.831489 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=87168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:33:55.832050 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.832345 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.832634 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.832923 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.833200 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.833483 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.835304 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.835608 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.835910 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.836205 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.836501 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.836784 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.853065 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=87167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:33:55.853438 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.853703 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.853967 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.856783 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=87168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:33:55.857163 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.857436 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.857704 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.862285 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.864025 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.864299 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.865703 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.867363 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=87167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:33:55.867449 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.867642 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.867726 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.867922 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.868201 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.868526 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.870809 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=87168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:33:55.871093 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.871378 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.871663 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.871954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.881947 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=87167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:33:55.882224 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.882501 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.882790 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.883075 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.883367 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.883643 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.885593 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=87168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:33:55.885881 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.886165 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.886451 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.886735 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.887047 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.887325 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.903304 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=87167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:33:55.907145 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=87167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:33:55.907801 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=87168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:33:55.910463 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.911155 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=87168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:33:55.912090 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.913719 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=87167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:33:55.914001 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.914285 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.914499 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.916237 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.917891 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=87168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:33:55.918173 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.918466 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.921870 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.923515 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=87167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:33:55.926082 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.926762 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.927065 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.927345 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.927617 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.927753 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=87168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:33:55.931030 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.931319 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.931605 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.931890 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.936864 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=87167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:33:55.937144 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.937417 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.941250 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=87168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:33:55.941539 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.941824 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.945351 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=87167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:33:55.945628 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.945904 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.946178 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.946447 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.946720 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.947021 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.947293 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.947559 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.947823 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.948094 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.948353 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.948621 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.948882 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=87167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.949785 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=87168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 22:33:55.950066 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.950889 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.951171 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.951448 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.951729 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.952014 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.952294 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.952568 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.952840 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.953135 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.953400 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.953670 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.953939 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=87168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 22:33:55.988298 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55d571c6d9c0 pid: 87167 (default app) Jul 21 22:33:55.991601 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea7a73-a7d8-4c10-b356-48954852ad6d None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:33:55.991894 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea7a73-a7d8-4c10-b356-48954852ad6d None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:33:55.992175 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea7a73-a7d8-4c10-b356-48954852ad6d None None] PATH_INFO: `/v3` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:33:55.993136 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55d571c6d9c0 pid: 87168 (default app) Jul 21 22:33:55.996337 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1/1] 10.222.0.52 () {50 vars in 791 bytes} [Thu Jul 21 22:33:55 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:04.583628 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa9b5f-6e42-4e75-aa38-bf9dc231a571 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:04.584163 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa9b5f-6e42-4e75-aa38-bf9dc231a571 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:04.584255 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa9b5f-6e42-4e75-aa38-bf9dc231a571 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:04.588583 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1/2] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:04 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:34:04.594115 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f68771-a3b6-4fb7-a80f-adfea24bf8f9 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:04.594115 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f68771-a3b6-4fb7-a80f-adfea24bf8f9 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:04.594568 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f68771-a3b6-4fb7-a80f-adfea24bf8f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:04.663260 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: INFO dbcounter [None req-80f68771-a3b6-4fb7-a80f-adfea24bf8f9 None None] Registered counter for database keystone Jul 21 22:34:04.684681 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG oslo_db.sqlalchemy.engines [None req-80f68771-a3b6-4fb7-a80f-adfea24bf8f9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=87167) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 21 22:34:04.690982 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writer thread running {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 21 22:34:04.872642 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG passlib.handlers.bcrypt [None req-80f68771-a3b6-4fb7-a80f-adfea24bf8f9 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=87167) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 21 22:34:04.873067 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG passlib.handlers.bcrypt [None req-80f68771-a3b6-4fb7-a80f-adfea24bf8f9 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=87167) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 21 22:34:04.926602 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-80f68771-a3b6-4fb7-a80f-adfea24bf8f9 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:04.928248 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-80f68771-a3b6-4fb7-a80f-adfea24bf8f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:05.050675 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 2/3] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:04 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 458 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 21 22:34:05.237297 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aa9b5f-6e42-4e75-aa38-bf9dc231a571 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:05.306699 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: INFO dbcounter [None req-14aa9b5f-6e42-4e75-aa38-bf9dc231a571 None None] Registered counter for database keystone Jul 21 22:34:05.326755 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG oslo_db.sqlalchemy.engines [None req-14aa9b5f-6e42-4e75-aa38-bf9dc231a571 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=87168) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 21 22:34:05.332920 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writer thread running {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 21 22:34:05.450715 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aa9b5f-6e42-4e75-aa38-bf9dc231a571 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:05.455156 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7e2c47-574e-4431-8961-b9bebf5ee23d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:05.455817 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e2c47-574e-4431-8961-b9bebf5ee23d admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:05.456104 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e2c47-574e-4431-8961-b9bebf5ee23d admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:05.456382 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e2c47-574e-4431-8961-b9bebf5ee23d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:05.464503 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7e2c47-574e-4431-8961-b9bebf5ee23d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:05.516703 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: WARNING keystone.common.rbac_enforcer.enforcer [None req-3a7e2c47-574e-4431-8961-b9bebf5ee23d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 21 22:34:05.517051 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7e2c47-574e-4431-8961-b9bebf5ee23d admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:05.526993 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 2/4] 10.222.0.52 () {58 vars in 1324 bytes} [Thu Jul 21 22:34:05 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 22:34:05.532897 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f68771-a3b6-4fb7-a80f-adfea24bf8f9 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:05.547909 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f68771-a3b6-4fb7-a80f-adfea24bf8f9 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:05.552408 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b219df-d315-4b5b-9145-606c2806f033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:05.553526 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b219df-d315-4b5b-9145-606c2806f033 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:05.553818 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b219df-d315-4b5b-9145-606c2806f033 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:05.554104 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b219df-d315-4b5b-9145-606c2806f033 admin admin] PATH_INFO: `/v3/projects/d47c52ce443d40eaa4e635852fb964cd` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:05.555620 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b219df-d315-4b5b-9145-606c2806f033 admin admin] RBAC: Authorizing `identity:get_project(project_id=d47c52ce443d40eaa4e635852fb964cd)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:05.608851 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: WARNING keystone.common.rbac_enforcer.enforcer [None req-28b219df-d315-4b5b-9145-606c2806f033 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 21 22:34:05.609391 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b219df-d315-4b5b-9145-606c2806f033 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:05.613879 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 3/5] 10.222.0.52 () {56 vars in 1207 bytes} [Thu Jul 21 22:34:05 2022] GET /identity/v3/projects/d47c52ce443d40eaa4e635852fb964cd => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:07.596512 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe2fe6e-9b3c-41c6-a7de-935f7d920515 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:07.597027 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe2fe6e-9b3c-41c6-a7de-935f7d920515 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:07.597116 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe2fe6e-9b3c-41c6-a7de-935f7d920515 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:07.601158 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 3/6] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:07.606718 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2379ac6b-1356-4aa4-8d3b-396757891848 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:07.607159 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2379ac6b-1356-4aa4-8d3b-396757891848 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:07.607313 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2379ac6b-1356-4aa4-8d3b-396757891848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:07.626248 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-2379ac6b-1356-4aa4-8d3b-396757891848 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:07.627818 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-2379ac6b-1356-4aa4-8d3b-396757891848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:07.635816 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 4/7] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:07 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 21 22:34:07.657868 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe2fe6e-9b3c-41c6-a7de-935f7d920515 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:07.669820 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe2fe6e-9b3c-41c6-a7de-935f7d920515 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:07.674019 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e1d307-2273-4155-b038-cb3c8f5c3b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:07.674646 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1d307-2273-4155-b038-cb3c8f5c3b18 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:07.674965 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1d307-2273-4155-b038-cb3c8f5c3b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:07.675252 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1d307-2273-4155-b038-cb3c8f5c3b18 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:07.682935 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e1d307-2273-4155-b038-cb3c8f5c3b18 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:07.685317 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e1d307-2273-4155-b038-cb3c8f5c3b18 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:07.693992 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 4/8] 10.222.0.52 () {58 vars in 1324 bytes} [Thu Jul 21 22:34:07 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 22:34:07.699058 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2379ac6b-1356-4aa4-8d3b-396757891848 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:07.708572 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2379ac6b-1356-4aa4-8d3b-396757891848 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:07.712980 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cac656f-e0b9-45f2-96eb-2e033b7ddfe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:07.713833 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac656f-e0b9-45f2-96eb-2e033b7ddfe8 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:07.714119 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac656f-e0b9-45f2-96eb-2e033b7ddfe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:07.714398 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac656f-e0b9-45f2-96eb-2e033b7ddfe8 admin admin] PATH_INFO: `/v3/users/031023a71a014731ad7c782eb926f065` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:07.715819 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cac656f-e0b9-45f2-96eb-2e033b7ddfe8 admin admin] RBAC: Authorizing `identity:get_user(user_id=031023a71a014731ad7c782eb926f065)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:07.717732 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cac656f-e0b9-45f2-96eb-2e033b7ddfe8 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:07.719318 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 5/9] 10.222.0.52 () {56 vars in 1198 bytes} [Thu Jul 21 22:34:07 2022] GET /identity/v3/users/031023a71a014731ad7c782eb926f065 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:09.782038 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980e04a-aa25-4462-9aa0-32da89da9bfd None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:09.782518 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980e04a-aa25-4462-9aa0-32da89da9bfd None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:09.782518 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980e04a-aa25-4462-9aa0-32da89da9bfd None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:09.783996 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 5/10] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:09.789159 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbee898-5c78-41b2-b66f-3a46ccdda777 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:09.789577 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbee898-5c78-41b2-b66f-3a46ccdda777 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:09.789726 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbee898-5c78-41b2-b66f-3a46ccdda777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:09.806677 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-cfbee898-5c78-41b2-b66f-3a46ccdda777 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:09.807891 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-cfbee898-5c78-41b2-b66f-3a46ccdda777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:09.819973 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428fa98-3fad-4895-9787-7f784cda7224 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:09.820257 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428fa98-3fad-4895-9787-7f784cda7224 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:09.820535 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428fa98-3fad-4895-9787-7f784cda7224 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:09.821884 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 6/11] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:34:09.827754 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc0961-ad24-4ce1-af24-c59646bdc285 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:09.828045 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc0961-ad24-4ce1-af24-c59646bdc285 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:09.828328 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc0961-ad24-4ce1-af24-c59646bdc285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:09.867803 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG passlib.handlers.bcrypt [None req-5efc0961-ad24-4ce1-af24-c59646bdc285 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=87168) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 21 22:34:09.868222 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG passlib.handlers.bcrypt [None req-5efc0961-ad24-4ce1-af24-c59646bdc285 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=87168) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 21 22:34:09.895310 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 6/12] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:34:09.914101 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-5efc0961-ad24-4ce1-af24-c59646bdc285 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:09.915714 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-5efc0961-ad24-4ce1-af24-c59646bdc285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:09.917696 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbee898-5c78-41b2-b66f-3a46ccdda777 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:09.941148 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:10.016858 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 7/13] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:34:10.021833 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbee898-5c78-41b2-b66f-3a46ccdda777 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:10.038435 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc0961-ad24-4ce1-af24-c59646bdc285 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:10.055752 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5494d1-2300-4cc6-b54b-23b235d8343b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:10.056551 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5494d1-2300-4cc6-b54b-23b235d8343b None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:10.056838 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5494d1-2300-4cc6-b54b-23b235d8343b None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:10.057121 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5494d1-2300-4cc6-b54b-23b235d8343b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:10.069094 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5494d1-2300-4cc6-b54b-23b235d8343b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:34:10.080180 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-ed5494d1-2300-4cc6-b54b-23b235d8343b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:34:10.080180 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:34:10.080180 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:34:10.080180 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:34:10.080180 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:34:10.080180 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:34:10.080180 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:34:10.080180 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:34:10.080180 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:34:10.080180 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:34:10.080180 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:34:10.080180 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:34:10.080180 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:34:10.080180 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:34:10.080180 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:34:10.080180 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:34:10.081185 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:34:10.081185 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:34:10.081185 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:34:10.081185 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:34:10.081185 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:34:10.081185 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:34:10.081185 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:34:10.081185 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:34:10.081185 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:34:10.081185 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:34:10.081185 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:34:10.081185 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:34:10.081185 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:34:10.081185 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:34:10.081185 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:34:10.081185 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:34:10.082115 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:34:10.082115 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:34:10.082115 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:34:10.082115 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:34:10.082115 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:34:10.082115 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:34:10.082115 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:34:10.082115 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:34:10.082115 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:34:10.082115 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:34:10.082115 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:34:10.082115 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:34:10.082115 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 7/14] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 22:34:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:34:10.087067 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5494d1-2300-4cc6-b54b-23b235d8343b None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:10.143425 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc0961-ad24-4ce1-af24-c59646bdc285 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:10.145701 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:10.169630 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:10.182190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ca994-0796-411e-b7d3-81fc79c11684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:10.183206 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ca994-0796-411e-b7d3-81fc79c11684 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:10.183686 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ca994-0796-411e-b7d3-81fc79c11684 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:10.184864 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ca994-0796-411e-b7d3-81fc79c11684 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:10.188949 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5494d1-2300-4cc6-b54b-23b235d8343b None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:10.222742 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c3dbfc-6e22-4b04-b137-81156c14481b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:10.223439 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c3dbfc-6e22-4b04-b137-81156c14481b None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:10.223731 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c3dbfc-6e22-4b04-b137-81156c14481b None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:10.224011 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c3dbfc-6e22-4b04-b137-81156c14481b None admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:10.224769 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c3dbfc-6e22-4b04-b137-81156c14481b None admin] RBAC: Adding query filter params (name=admin) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:10.225079 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c3dbfc-6e22-4b04-b137-81156c14481b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:10.226517 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c3dbfc-6e22-4b04-b137-81156c14481b None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:10.239038 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 8/15] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:34:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:34:10.244549 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c3dbfc-6e22-4b04-b137-81156c14481b None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:10.253010 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ca994-0796-411e-b7d3-81fc79c11684 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:10.254536 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ca994-0796-411e-b7d3-81fc79c11684 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:10.347865 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c3dbfc-6e22-4b04-b137-81156c14481b None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:10.353307 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 8/16] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:34:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 22:34:10.360278 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ca994-0796-411e-b7d3-81fc79c11684 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:10.362992 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:10.382617 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd1bbf4-f5cc-4770-99b2-218e31efccb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:10.383534 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd1bbf4-f5cc-4770-99b2-218e31efccb1 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:10.383819 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd1bbf4-f5cc-4770-99b2-218e31efccb1 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:10.384091 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd1bbf4-f5cc-4770-99b2-218e31efccb1 None admin] PATH_INFO: `/v3/users/031023a71a014731ad7c782eb926f065` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:10.385309 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd1bbf4-f5cc-4770-99b2-218e31efccb1 None admin] RBAC: Authorizing `identity:get_user(user_id=031023a71a014731ad7c782eb926f065)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:10.386802 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd1bbf4-f5cc-4770-99b2-218e31efccb1 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:10.388440 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 9/17] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 22:34:10 2022] GET /identity/v3/users/031023a71a014731ad7c782eb926f065 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:34:10.393772 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd1bbf4-f5cc-4770-99b2-218e31efccb1 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:10.408643 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:10.458317 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ca994-0796-411e-b7d3-81fc79c11684 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:10.493288 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0cfaf8-4dbd-446f-870f-678861e4846b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:10.493742 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd1bbf4-f5cc-4770-99b2-218e31efccb1 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:10.494087 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0cfaf8-4dbd-446f-870f-678861e4846b None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:10.494371 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0cfaf8-4dbd-446f-870f-678861e4846b None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:10.494650 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0cfaf8-4dbd-446f-870f-678861e4846b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:10.514339 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0cfaf8-4dbd-446f-870f-678861e4846b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:10.515829 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0cfaf8-4dbd-446f-870f-678861e4846b None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:10.527040 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce1900e-fd84-4b65-9f08-4d60a4225c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:10.528352 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce1900e-fd84-4b65-9f08-4d60a4225c46 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:10.528637 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce1900e-fd84-4b65-9f08-4d60a4225c46 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:10.528913 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce1900e-fd84-4b65-9f08-4d60a4225c46 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:10.530099 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce1900e-fd84-4b65-9f08-4d60a4225c46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:10.531557 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce1900e-fd84-4b65-9f08-4d60a4225c46 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:10.533167 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 10/18] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:34:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:34:10.536560 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-fb0cfaf8-4dbd-446f-870f-678861e4846b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:34:10.536560 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:34:10.536560 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:34:10.536560 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:34:10.536560 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:34:10.536560 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:34:10.536560 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:34:10.536560 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:34:10.536560 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:34:10.536560 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:34:10.536560 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:34:10.536560 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:34:10.536560 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:34:10.536560 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:34:10.536560 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:34:10.537530 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:34:10.537530 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:34:10.537530 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:34:10.537530 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:34:10.537530 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:34:10.537530 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:34:10.537530 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:34:10.537530 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:34:10.537530 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:34:10.537530 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:34:10.537530 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:34:10.537530 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:34:10.537530 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:34:10.537530 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:34:10.537530 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:34:10.537530 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:34:10.537530 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:34:10.538377 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:34:10.538377 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:34:10.538377 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:34:10.538377 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:34:10.538377 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 9/19] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:34:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:34:10.539104 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce1900e-fd84-4b65-9f08-4d60a4225c46 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:10.544681 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0cfaf8-4dbd-446f-870f-678861e4846b None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:10.600204 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:10.618527 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:10.636907 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce1900e-fd84-4b65-9f08-4d60a4225c46 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:10.645609 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0cfaf8-4dbd-446f-870f-678861e4846b None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:10.670983 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f4e530-3498-4325-b32e-b62915c79680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:10.671641 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f4e530-3498-4325-b32e-b62915c79680 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:10.671925 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f4e530-3498-4325-b32e-b62915c79680 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:10.672198 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f4e530-3498-4325-b32e-b62915c79680 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:10.673033 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f4e530-3498-4325-b32e-b62915c79680 None admin] RBAC: Adding query filter params (role.id=d1ddee590a964ff1bbb4ee8716173021, scope.domain.id=default, user.id=031023a71a014731ad7c782eb926f065) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:10.673345 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f4e530-3498-4325-b32e-b62915c79680 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:10.674799 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f4e530-3498-4325-b32e-b62915c79680 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:10.680069 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ba6b09-0a50-4397-bf71-2df655bafd41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:10.680736 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba6b09-0a50-4397-bf71-2df655bafd41 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:10.681020 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba6b09-0a50-4397-bf71-2df655bafd41 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:10.681305 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba6b09-0a50-4397-bf71-2df655bafd41 None admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:10.681417 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 11/20] 10.222.0.52 () {56 vars in 1322 bytes} [Thu Jul 21 22:34:10 2022] GET /identity/v3/role_assignments?user.id=031023a71a014731ad7c782eb926f065&scope.domain.id=default&role.id=d1ddee590a964ff1bbb4ee8716173021 => generated 233 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:34:10.682135 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ba6b09-0a50-4397-bf71-2df655bafd41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:10.682494 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ba6b09-0a50-4397-bf71-2df655bafd41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:10.684522 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ba6b09-0a50-4397-bf71-2df655bafd41 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:10.704635 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 10/21] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:34:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:34:12.999843 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2aa3ca-4c83-4ce0-90a6-097dd0b584b8 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:13.000344 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2aa3ca-4c83-4ce0-90a6-097dd0b584b8 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:13.000440 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2aa3ca-4c83-4ce0-90a6-097dd0b584b8 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:13.001759 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 12/22] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:13.007128 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e955c98-1a72-4618-a746-eb7e20e62228 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:13.007550 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e955c98-1a72-4618-a746-eb7e20e62228 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:13.007729 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e955c98-1a72-4618-a746-eb7e20e62228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:13.031829 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-6e955c98-1a72-4618-a746-eb7e20e62228 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:13.033056 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-6e955c98-1a72-4618-a746-eb7e20e62228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:13.037768 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674a2d6-3ad4-49d2-afab-d611cfebb86e None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:13.038056 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674a2d6-3ad4-49d2-afab-d611cfebb86e None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:13.038337 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674a2d6-3ad4-49d2-afab-d611cfebb86e None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:13.039724 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 13/23] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:34:13.045126 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a5701-0318-4c8d-97c7-bc81a485029e None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:13.045416 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a5701-0318-4c8d-97c7-bc81a485029e None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:13.045697 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a5701-0318-4c8d-97c7-bc81a485029e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:13.068598 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-227a5701-0318-4c8d-97c7-bc81a485029e None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:13.070660 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-227a5701-0318-4c8d-97c7-bc81a485029e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:13.201267 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 11/24] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:34:13.234793 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 14/25] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:34:13.250110 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e955c98-1a72-4618-a746-eb7e20e62228 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:13.290369 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a5701-0318-4c8d-97c7-bc81a485029e None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:13.331204 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:13.421391 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e955c98-1a72-4618-a746-eb7e20e62228 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:13.469419 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:13.488540 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71edb563-e15b-4207-956b-c95feae1039a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:13.489357 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71edb563-e15b-4207-956b-c95feae1039a None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:13.491910 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71edb563-e15b-4207-956b-c95feae1039a None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:13.491910 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71edb563-e15b-4207-956b-c95feae1039a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:13.524654 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71edb563-e15b-4207-956b-c95feae1039a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:13.524654 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71edb563-e15b-4207-956b-c95feae1039a None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:13.534869 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a5701-0318-4c8d-97c7-bc81a485029e None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:13.553869 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-71edb563-e15b-4207-956b-c95feae1039a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:34:13.553869 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:34:13.553869 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:34:13.553869 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:34:13.553869 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:34:13.553869 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:34:13.553869 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:34:13.553869 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:34:13.553869 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:34:13.553869 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:34:13.553869 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:34:13.553869 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:34:13.553869 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:34:13.553869 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:34:13.553869 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:34:13.561489 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:34:13.561489 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:34:13.561489 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:34:13.561489 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:34:13.561489 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:34:13.561489 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:34:13.561489 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:34:13.561489 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:34:13.561489 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:34:13.561489 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:34:13.561489 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:34:13.561489 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:34:13.561489 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:34:13.561489 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:34:13.561489 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:34:13.561489 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:34:13.561489 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:34:13.562509 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:34:13.562509 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:34:13.562509 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:34:13.562509 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:34:13.562509 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 12/26] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:34:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:34:13.562509 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097c6a2-29a4-4377-a0ee-1e9f5e3a608b None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:13.562509 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097c6a2-29a4-4377-a0ee-1e9f5e3a608b None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:13.562509 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097c6a2-29a4-4377-a0ee-1e9f5e3a608b None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:13.562509 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 13/27] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:13.567794 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5097c6a2-29a4-4377-a0ee-1e9f5e3a608b None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:13.608070 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131696ec-bd10-4e62-9fec-1b3aabdecf52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:13.610095 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131696ec-bd10-4e62-9fec-1b3aabdecf52 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:13.610923 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131696ec-bd10-4e62-9fec-1b3aabdecf52 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:13.612307 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131696ec-bd10-4e62-9fec-1b3aabdecf52 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:13.629388 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131696ec-bd10-4e62-9fec-1b3aabdecf52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:34:13.638321 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-131696ec-bd10-4e62-9fec-1b3aabdecf52 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:34:13.638321 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:34:13.638321 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:34:13.638321 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:34:13.638321 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:34:13.638321 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:34:13.638321 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:34:13.638321 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:34:13.638321 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:34:13.638321 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:34:13.638321 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:34:13.638321 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:34:13.638321 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:34:13.638321 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:34:13.638321 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:34:13.638321 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:34:13.639397 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:34:13.639397 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:34:13.639397 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:34:13.639397 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:34:13.639397 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:34:13.639397 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:34:13.639397 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:34:13.639397 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:34:13.639397 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:34:13.639397 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:34:13.639397 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:34:13.639397 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:34:13.639397 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:34:13.639397 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:34:13.639397 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:34:13.639397 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:34:13.640387 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:34:13.640387 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:34:13.640387 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:34:13.640387 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:34:13.640387 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:34:13.640387 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:34:13.640387 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:34:13.640387 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:34:13.640387 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:34:13.640387 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:34:13.640387 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:34:13.640387 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:34:13.641745 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 15/28] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 22:34:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:34:13.643183 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45a250-a996-469d-bdb3-ab6346e7d6de None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:13.643548 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45a250-a996-469d-bdb3-ab6346e7d6de None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:13.643898 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45a250-a996-469d-bdb3-ab6346e7d6de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:13.661576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-0a45a250-a996-469d-bdb3-ab6346e7d6de None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:13.662992 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-0a45a250-a996-469d-bdb3-ab6346e7d6de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:13.715245 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:13.720198 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5097c6a2-29a4-4377-a0ee-1e9f5e3a608b None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:13.758296 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 16/29] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:34:13.767803 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c0c52-b72d-4ffb-97ce-5dffbf587662 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:13.768092 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c0c52-b72d-4ffb-97ce-5dffbf587662 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:13.768373 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c0c52-b72d-4ffb-97ce-5dffbf587662 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:13.769354 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e9b826-3378-49de-8c19-07dcc9ef94cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:13.770017 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9b826-3378-49de-8c19-07dcc9ef94cd None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:13.770296 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9b826-3378-49de-8c19-07dcc9ef94cd None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:13.770568 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9b826-3378-49de-8c19-07dcc9ef94cd None admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:13.772046 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e9b826-3378-49de-8c19-07dcc9ef94cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:13.772359 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e9b826-3378-49de-8c19-07dcc9ef94cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:13.773477 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 17/30] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:13 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:13.774261 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265c0c52-b72d-4ffb-97ce-5dffbf587662 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:13.775539 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e9b826-3378-49de-8c19-07dcc9ef94cd None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:13.795659 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 14/31] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:34:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:13.797077 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b41b8-2b4e-44f1-ab29-f29aab885a9e None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:13.797366 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b41b8-2b4e-44f1-ab29-f29aab885a9e None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:13.797645 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b41b8-2b4e-44f1-ab29-f29aab885a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:13.817560 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-cb2b41b8-2b4e-44f1-ab29-f29aab885a9e None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:13.818894 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-cb2b41b8-2b4e-44f1-ab29-f29aab885a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:13.863955 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:13.896188 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 15/32] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:34:13.897099 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b41b8-2b4e-44f1-ab29-f29aab885a9e None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:13.931927 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265c0c52-b72d-4ffb-97ce-5dffbf587662 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:13.980881 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71922d6d-b982-4230-834a-97a33ba2d93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:13.981557 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71922d6d-b982-4230-834a-97a33ba2d93e None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:13.981848 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71922d6d-b982-4230-834a-97a33ba2d93e None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:13.982126 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71922d6d-b982-4230-834a-97a33ba2d93e None admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:13.983193 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71922d6d-b982-4230-834a-97a33ba2d93e None admin] RBAC: Adding query filter params (name=admin) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:13.984092 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71922d6d-b982-4230-834a-97a33ba2d93e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:13.985792 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71922d6d-b982-4230-834a-97a33ba2d93e None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:13.997224 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 18/33] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:34:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:13.998268 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71922d6d-b982-4230-834a-97a33ba2d93e None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:14.007593 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:14.021087 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b41b8-2b4e-44f1-ab29-f29aab885a9e None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:14.067983 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe51c324-9915-4e8c-830f-1266c296e0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:14.068588 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe51c324-9915-4e8c-830f-1266c296e0f7 None admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:14.069137 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe51c324-9915-4e8c-830f-1266c296e0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:14.069422 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe51c324-9915-4e8c-830f-1266c296e0f7 None admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:14.070888 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe51c324-9915-4e8c-830f-1266c296e0f7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:14.072971 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe51c324-9915-4e8c-830f-1266c296e0f7 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:14.118240 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71922d6d-b982-4230-834a-97a33ba2d93e None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:14.145355 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:14.164474 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044d1046-e99f-4e5f-938c-82d365576db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:14.170904 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d1046-e99f-4e5f-938c-82d365576db0 None admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:14.170904 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d1046-e99f-4e5f-938c-82d365576db0 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:14.170904 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d1046-e99f-4e5f-938c-82d365576db0 None admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:14.170904 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044d1046-e99f-4e5f-938c-82d365576db0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:14.170904 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044d1046-e99f-4e5f-938c-82d365576db0 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:14.266481 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 16/34] 10.222.0.52 () {60 vars in 1131 bytes} [Thu Jul 21 22:34:13 2022] POST /identity/v3/roles => generated 219 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:14.267653 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe51c324-9915-4e8c-830f-1266c296e0f7 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:14.458751 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 19/35] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jul 21 22:34:13 2022] POST /identity/v3/projects => generated 301 bytes in 461 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:14.460128 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044d1046-e99f-4e5f-938c-82d365576db0 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:14.470256 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe51c324-9915-4e8c-830f-1266c296e0f7 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:14.524092 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d2d41a-7d83-4f00-8e52-caaecc6cf144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:14.524847 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d2d41a-7d83-4f00-8e52-caaecc6cf144 None admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:14.527877 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d2d41a-7d83-4f00-8e52-caaecc6cf144 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:14.527877 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d2d41a-7d83-4f00-8e52-caaecc6cf144 None admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:14.527877 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d2d41a-7d83-4f00-8e52-caaecc6cf144 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:14.528642 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d2d41a-7d83-4f00-8e52-caaecc6cf144 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:14.607475 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044d1046-e99f-4e5f-938c-82d365576db0 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:14.651894 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 17/36] 10.222.0.52 () {60 vars in 1131 bytes} [Thu Jul 21 22:34:14 2022] POST /identity/v3/roles => generated 225 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:14.653155 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb1f1d8-20db-4636-9d29-8fe5dba6a573 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:14.653700 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb1f1d8-20db-4636-9d29-8fe5dba6a573 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:14.654203 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb1f1d8-20db-4636-9d29-8fe5dba6a573 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:14.659194 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 18/37] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:14 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:14.662902 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d98873c-f41d-41ec-95cc-1c255c9bbe86 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:14.668838 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d98873c-f41d-41ec-95cc-1c255c9bbe86 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:14.668838 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d98873c-f41d-41ec-95cc-1c255c9bbe86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:14.690647 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-6d98873c-f41d-41ec-95cc-1c255c9bbe86 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:14.692873 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-6d98873c-f41d-41ec-95cc-1c255c9bbe86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:14.714719 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:14.761517 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae330b2e-d93e-4d64-8fde-69c9bb7ab260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:14.762543 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae330b2e-d93e-4d64-8fde-69c9bb7ab260 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:14.763320 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae330b2e-d93e-4d64-8fde-69c9bb7ab260 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:14.763320 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae330b2e-d93e-4d64-8fde-69c9bb7ab260 None admin] PATH_INFO: `/v3/users/031023a71a014731ad7c782eb926f065` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:14.768175 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae330b2e-d93e-4d64-8fde-69c9bb7ab260 None admin] RBAC: Authorizing `identity:get_user(user_id=031023a71a014731ad7c782eb926f065)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:14.768175 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae330b2e-d93e-4d64-8fde-69c9bb7ab260 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:14.769716 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 20/38] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 22:34:14 2022] GET /identity/v3/users/031023a71a014731ad7c782eb926f065 => generated 247 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:14.773603 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3397cba-7609-4a4c-b4f6-586bf78bfeb7 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:14.773603 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3397cba-7609-4a4c-b4f6-586bf78bfeb7 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:14.773603 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3397cba-7609-4a4c-b4f6-586bf78bfeb7 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:14.773603 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 21/39] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:14.779169 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3397cba-7609-4a4c-b4f6-586bf78bfeb7 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:14.792783 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 19/40] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:34:14.794707 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db94031-fe57-42dc-b499-dbe7bc59942b None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:14.794884 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db94031-fe57-42dc-b499-dbe7bc59942b None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:14.797011 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db94031-fe57-42dc-b499-dbe7bc59942b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:14.817804 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-2db94031-fe57-42dc-b499-dbe7bc59942b None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:14.819184 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-2db94031-fe57-42dc-b499-dbe7bc59942b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:14.836407 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:14.900600 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 20/41] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:34:14.901566 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db94031-fe57-42dc-b499-dbe7bc59942b None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:14.903802 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3397cba-7609-4a4c-b4f6-586bf78bfeb7 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:14.939181 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8241837e-65e9-4e59-a4c7-372a45841bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:14.939979 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241837e-65e9-4e59-a4c7-372a45841bb3 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:14.940268 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241837e-65e9-4e59-a4c7-372a45841bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:14.940548 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241837e-65e9-4e59-a4c7-372a45841bb3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:14.941810 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8241837e-65e9-4e59-a4c7-372a45841bb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:14.943297 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8241837e-65e9-4e59-a4c7-372a45841bb3 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:14.945068 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 22/42] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:34:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:34:14.945659 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8241837e-65e9-4e59-a4c7-372a45841bb3 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:14.971238 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:15.006193 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db94031-fe57-42dc-b499-dbe7bc59942b None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:15.041751 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1576f90-b5fb-4a6f-834e-72e74907aef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:15.042438 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1576f90-b5fb-4a6f-834e-72e74907aef1 None admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:15.042733 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1576f90-b5fb-4a6f-834e-72e74907aef1 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:15.043686 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1576f90-b5fb-4a6f-834e-72e74907aef1 None admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:15.044440 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1576f90-b5fb-4a6f-834e-72e74907aef1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:15.045841 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1576f90-b5fb-4a6f-834e-72e74907aef1 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:15.046912 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8241837e-65e9-4e59-a4c7-372a45841bb3 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:15.061459 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:15.080914 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bfe2c7-4c34-4d07-be61-5f4557fe3ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:15.081738 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bfe2c7-4c34-4d07-be61-5f4557fe3ab8 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:15.082027 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bfe2c7-4c34-4d07-be61-5f4557fe3ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:15.082308 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bfe2c7-4c34-4d07-be61-5f4557fe3ab8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:15.083493 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bfe2c7-4c34-4d07-be61-5f4557fe3ab8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:15.084939 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bfe2c7-4c34-4d07-be61-5f4557fe3ab8 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:15.086611 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 23/43] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:34:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:15.087413 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bfe2c7-4c34-4d07-be61-5f4557fe3ab8 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:15.114291 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 21/44] 10.222.0.52 () {60 vars in 1131 bytes} [Thu Jul 21 22:34:14 2022] POST /identity/v3/roles => generated 223 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:15.115286 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1576f90-b5fb-4a6f-834e-72e74907aef1 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:15.200810 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bfe2c7-4c34-4d07-be61-5f4557fe3ab8 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:15.239328 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d29554-a793-419c-a20d-bfd20e369c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:15.239538 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1576f90-b5fb-4a6f-834e-72e74907aef1 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:15.240149 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d29554-a793-419c-a20d-bfd20e369c9c None admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:15.240442 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d29554-a793-419c-a20d-bfd20e369c9c None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:15.240730 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d29554-a793-419c-a20d-bfd20e369c9c None admin] PATH_INFO: `/v3/domains/default/users/031023a71a014731ad7c782eb926f065/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:15.242912 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d29554-a793-419c-a20d-bfd20e369c9c None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=031023a71a014731ad7c782eb926f065, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:15.244417 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d29554-a793-419c-a20d-bfd20e369c9c None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:15.266807 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:15.275054 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc9967d-129b-4eb0-a128-f74abb8b34cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:15.275748 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc9967d-129b-4eb0-a128-f74abb8b34cb None admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:15.276034 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc9967d-129b-4eb0-a128-f74abb8b34cb None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:15.276721 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc9967d-129b-4eb0-a128-f74abb8b34cb None admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:15.278012 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc9967d-129b-4eb0-a128-f74abb8b34cb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:15.279490 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc9967d-129b-4eb0-a128-f74abb8b34cb None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:15.327886 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 24/45] 10.222.0.52 () {58 vars in 1361 bytes} [Thu Jul 21 22:34:15 2022] PUT /identity/v3/domains/default/users/031023a71a014731ad7c782eb926f065/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:15.375465 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 22/46] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jul 21 22:34:15 2022] POST /identity/v3/projects => generated 312 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:17.365478 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fd4aac-15c1-4af1-8625-92566ae320fc None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:17.365985 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fd4aac-15c1-4af1-8625-92566ae320fc None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:17.366074 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fd4aac-15c1-4af1-8625-92566ae320fc None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:17.367456 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 25/47] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:17.372943 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b52c55-d940-4d5b-b6f2-a25633ea959d None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:17.373342 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b52c55-d940-4d5b-b6f2-a25633ea959d None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:17.373530 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b52c55-d940-4d5b-b6f2-a25633ea959d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:17.390085 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-a3b52c55-d940-4d5b-b6f2-a25633ea959d None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:17.391313 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-a3b52c55-d940-4d5b-b6f2-a25633ea959d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:17.423218 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c930b5-dc94-4ad7-a467-55eb02391a26 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:17.423517 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c930b5-dc94-4ad7-a467-55eb02391a26 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:17.423802 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c930b5-dc94-4ad7-a467-55eb02391a26 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:17.425173 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 26/48] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:17.429989 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57060832-15ad-4b2c-a72c-2974ce810762 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:17.430280 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57060832-15ad-4b2c-a72c-2974ce810762 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:17.430564 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57060832-15ad-4b2c-a72c-2974ce810762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:17.446940 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-57060832-15ad-4b2c-a72c-2974ce810762 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:17.448461 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-57060832-15ad-4b2c-a72c-2974ce810762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:17.458409 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 23/49] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:34:17.482422 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b52c55-d940-4d5b-b6f2-a25633ea959d None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:17.485298 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:17.517018 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 27/50] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:34:17.541012 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57060832-15ad-4b2c-a72c-2974ce810762 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:17.582844 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b52c55-d940-4d5b-b6f2-a25633ea959d None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:17.594091 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:17.617909 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5783135e-9cd9-4cf3-8199-60d06daa3b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:17.618639 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5783135e-9cd9-4cf3-8199-60d06daa3b95 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:17.618965 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5783135e-9cd9-4cf3-8199-60d06daa3b95 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:17.619539 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5783135e-9cd9-4cf3-8199-60d06daa3b95 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:17.631949 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5783135e-9cd9-4cf3-8199-60d06daa3b95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:34:17.641431 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-5783135e-9cd9-4cf3-8199-60d06daa3b95 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:34:17.641431 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:34:17.641431 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:34:17.641431 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:34:17.641431 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:34:17.641431 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:34:17.641431 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:34:17.641431 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:34:17.641431 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:34:17.641431 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:34:17.641431 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:34:17.641431 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:34:17.641431 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:34:17.641431 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:34:17.641431 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:34:17.641431 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:34:17.643516 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:34:17.643516 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:34:17.643516 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:34:17.643516 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:34:17.643516 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:34:17.643516 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:34:17.643516 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:34:17.643516 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:34:17.643516 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:34:17.643516 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:34:17.643516 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:34:17.643516 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:34:17.643516 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:34:17.643516 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:34:17.643516 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:34:17.643516 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:34:17.644281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:34:17.644281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:34:17.644281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:34:17.644281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:34:17.644281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:34:17.644281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:34:17.644281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:34:17.644281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:34:17.644281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:34:17.644281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:34:17.644281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:34:17.644281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:34:17.644843 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57060832-15ad-4b2c-a72c-2974ce810762 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:17.644910 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 24/51] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 22:34:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:34:17.648475 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5783135e-9cd9-4cf3-8199-60d06daa3b95 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:17.679104 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71384f19-834c-48a2-9b19-0fd55f673ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:17.679912 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71384f19-834c-48a2-9b19-0fd55f673ab0 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:17.680347 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71384f19-834c-48a2-9b19-0fd55f673ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:17.680458 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71384f19-834c-48a2-9b19-0fd55f673ab0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:17.681646 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71384f19-834c-48a2-9b19-0fd55f673ab0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:17.683134 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71384f19-834c-48a2-9b19-0fd55f673ab0 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:17.684909 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 28/52] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:34:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:34:17.690136 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71384f19-834c-48a2-9b19-0fd55f673ab0 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:17.731984 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:17.749117 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5783135e-9cd9-4cf3-8199-60d06daa3b95 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:17.783359 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180d36a9-db53-47b8-9b93-a934c2754ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:17.784026 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d36a9-db53-47b8-9b93-a934c2754ae0 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:17.784309 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d36a9-db53-47b8-9b93-a934c2754ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:17.784590 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d36a9-db53-47b8-9b93-a934c2754ae0 None admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:17.785334 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180d36a9-db53-47b8-9b93-a934c2754ae0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:17.785643 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180d36a9-db53-47b8-9b93-a934c2754ae0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:17.787048 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180d36a9-db53-47b8-9b93-a934c2754ae0 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:17.790563 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71384f19-834c-48a2-9b19-0fd55f673ab0 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:17.827706 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450775d3-4593-4ba1-b76e-9e2aa82c0bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:17.828392 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450775d3-4593-4ba1-b76e-9e2aa82c0bd5 None admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:17.828688 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450775d3-4593-4ba1-b76e-9e2aa82c0bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:17.829098 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450775d3-4593-4ba1-b76e-9e2aa82c0bd5 None admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:17.829709 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450775d3-4593-4ba1-b76e-9e2aa82c0bd5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:17.831278 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450775d3-4593-4ba1-b76e-9e2aa82c0bd5 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:17.904286 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 25/53] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:34:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:34:17.909884 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180d36a9-db53-47b8-9b93-a934c2754ae0 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:17.940512 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:17.949500 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 29/54] 10.222.0.52 () {60 vars in 1140 bytes} [Thu Jul 21 22:34:17 2022] POST /identity/v3/projects => generated 298 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:34:18.016195 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180d36a9-db53-47b8-9b93-a934c2754ae0 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:18.051021 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ee58a-574e-456b-8fda-360b6f6086ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:18.051926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ee58a-574e-456b-8fda-360b6f6086ba None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:18.052212 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ee58a-574e-456b-8fda-360b6f6086ba None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:18.052492 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ee58a-574e-456b-8fda-360b6f6086ba None admin] PATH_INFO: `/v3/users/031023a71a014731ad7c782eb926f065` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:18.053987 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1ee58a-574e-456b-8fda-360b6f6086ba None admin] RBAC: Authorizing `identity:get_user(user_id=031023a71a014731ad7c782eb926f065)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:18.055518 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1ee58a-574e-456b-8fda-360b6f6086ba None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:18.057224 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 26/55] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 22:34:17 2022] GET /identity/v3/users/031023a71a014731ad7c782eb926f065 => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:34:18.063469 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450775d3-4593-4ba1-b76e-9e2aa82c0bd5 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:18.158368 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:18.166944 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450775d3-4593-4ba1-b76e-9e2aa82c0bd5 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:18.201980 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724e7417-ffee-4b93-9516-7d1add261f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:18.202842 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e7417-ffee-4b93-9516-7d1add261f10 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:18.203136 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e7417-ffee-4b93-9516-7d1add261f10 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:18.203418 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e7417-ffee-4b93-9516-7d1add261f10 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:18.204595 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724e7417-ffee-4b93-9516-7d1add261f10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:18.206081 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724e7417-ffee-4b93-9516-7d1add261f10 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:18.207743 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 30/56] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:34:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:18.213550 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ee58a-574e-456b-8fda-360b6f6086ba None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:18.310126 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:18.313149 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ee58a-574e-456b-8fda-360b6f6086ba None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:18.345850 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceea481-fc44-4ab6-9460-0abe87953fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:18.346513 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceea481-fc44-4ab6-9460-0abe87953fe1 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:18.346830 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceea481-fc44-4ab6-9460-0abe87953fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:18.347116 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceea481-fc44-4ab6-9460-0abe87953fe1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:18.347957 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceea481-fc44-4ab6-9460-0abe87953fe1 None admin] RBAC: Adding query filter params (role.id=d1ddee590a964ff1bbb4ee8716173021, scope.domain.id=default, user.id=031023a71a014731ad7c782eb926f065) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:18.348273 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceea481-fc44-4ab6-9460-0abe87953fe1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:18.349712 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceea481-fc44-4ab6-9460-0abe87953fe1 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:18.359282 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 27/57] 10.222.0.52 () {56 vars in 1322 bytes} [Thu Jul 21 22:34:18 2022] GET /identity/v3/role_assignments?user.id=031023a71a014731ad7c782eb926f065&scope.domain.id=default&role.id=d1ddee590a964ff1bbb4ee8716173021 => generated 530 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:19.987209 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc6cc2b-eaab-465a-a60e-c612d8d2eb52 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:19.987716 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc6cc2b-eaab-465a-a60e-c612d8d2eb52 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:19.987827 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc6cc2b-eaab-465a-a60e-c612d8d2eb52 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:19.989157 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 31/58] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:19.994262 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a15240-3f58-49fa-8284-d5cfe16ffd38 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:19.994661 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a15240-3f58-49fa-8284-d5cfe16ffd38 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:19.994861 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a15240-3f58-49fa-8284-d5cfe16ffd38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:20.011889 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-f7a15240-3f58-49fa-8284-d5cfe16ffd38 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:20.013069 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-f7a15240-3f58-49fa-8284-d5cfe16ffd38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:20.077519 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 28/59] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:34:20.100705 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc6cc2b-eaab-465a-a60e-c612d8d2eb52 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:20.199741 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc6cc2b-eaab-465a-a60e-c612d8d2eb52 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:20.233693 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ebb432-c33f-4aa4-842a-848f55972179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:20.234483 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebb432-c33f-4aa4-842a-848f55972179 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:20.234793 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebb432-c33f-4aa4-842a-848f55972179 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:20.235073 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebb432-c33f-4aa4-842a-848f55972179 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:20.236243 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ebb432-c33f-4aa4-842a-848f55972179 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:20.237770 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ebb432-c33f-4aa4-842a-848f55972179 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:20.239378 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 32/60] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:34:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:20.244872 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a15240-3f58-49fa-8284-d5cfe16ffd38 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:20.334234 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:20.341080 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a15240-3f58-49fa-8284-d5cfe16ffd38 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:20.373474 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d519b9-cda8-469c-9afd-bc3d57f4e1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:20.374134 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d519b9-cda8-469c-9afd-bc3d57f4e1f7 None admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:20.374417 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d519b9-cda8-469c-9afd-bc3d57f4e1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:20.374692 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d519b9-cda8-469c-9afd-bc3d57f4e1f7 None admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:20.375427 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d519b9-cda8-469c-9afd-bc3d57f4e1f7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:20.376859 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d519b9-cda8-469c-9afd-bc3d57f4e1f7 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:20.455784 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 29/61] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 21 22:34:20 2022] POST /identity/v3/users => generated 275 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:23.616849 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba9ef5-686c-46f4-8bb5-5fc3e6a0e02c None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:23.616849 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba9ef5-686c-46f4-8bb5-5fc3e6a0e02c None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:23.616849 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba9ef5-686c-46f4-8bb5-5fc3e6a0e02c None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:23.616849 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 33/62] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:23.622847 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0deeced-9814-4039-9e44-21cf8ca1d2eb None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:23.623229 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0deeced-9814-4039-9e44-21cf8ca1d2eb None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:23.623407 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0deeced-9814-4039-9e44-21cf8ca1d2eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:23.633649 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334b063-4a30-4042-8678-4da5ffaf6256 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:23.634204 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334b063-4a30-4042-8678-4da5ffaf6256 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:23.634709 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334b063-4a30-4042-8678-4da5ffaf6256 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:23.636361 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 34/63] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:23.641479 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b3e3d-338f-45c8-aeb6-44ab6a5b37f1 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:23.642000 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b3e3d-338f-45c8-aeb6-44ab6a5b37f1 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:23.642479 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b3e3d-338f-45c8-aeb6-44ab6a5b37f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:23.646281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-a0deeced-9814-4039-9e44-21cf8ca1d2eb None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:23.647517 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-a0deeced-9814-4039-9e44-21cf8ca1d2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:23.675816 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-c03b3e3d-338f-45c8-aeb6-44ab6a5b37f1 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:23.677019 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-c03b3e3d-338f-45c8-aeb6-44ab6a5b37f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:23.727136 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:23.772831 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 30/64] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:34:23.800453 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 35/65] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:34:23.803689 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0deeced-9814-4039-9e44-21cf8ca1d2eb None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:23.828343 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b3e3d-338f-45c8-aeb6-44ab6a5b37f1 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:23.911301 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:23.935167 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b3e3d-338f-45c8-aeb6-44ab6a5b37f1 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:23.936003 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0deeced-9814-4039-9e44-21cf8ca1d2eb None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:23.980526 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6641c118-5b88-4210-8752-7a493a2c49d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:23.981240 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9edb611-260b-4719-8b5b-6d2d232df9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:23.981365 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6641c118-5b88-4210-8752-7a493a2c49d3 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:23.981622 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6641c118-5b88-4210-8752-7a493a2c49d3 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:23.981907 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6641c118-5b88-4210-8752-7a493a2c49d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:23.982594 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9edb611-260b-4719-8b5b-6d2d232df9a6 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:23.983057 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9edb611-260b-4719-8b5b-6d2d232df9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:23.983472 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9edb611-260b-4719-8b5b-6d2d232df9a6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:23.994449 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9edb611-260b-4719-8b5b-6d2d232df9a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:34:23.994646 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6641c118-5b88-4210-8752-7a493a2c49d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:34:24.002760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-6641c118-5b88-4210-8752-7a493a2c49d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:34:24.002760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:34:24.002760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:34:24.002760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:34:24.002760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:34:24.002760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:34:24.002760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:34:24.002760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:34:24.002760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:34:24.002760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:34:24.002760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:34:24.002760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:34:24.002760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:34:24.002760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:34:24.002760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:34:24.002760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:34:24.003690 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:34:24.003690 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:34:24.003690 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:34:24.003690 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:34:24.003690 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:34:24.003690 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:34:24.003690 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:34:24.003690 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:34:24.003690 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:34:24.003690 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:34:24.003690 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:34:24.003690 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:34:24.003690 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:34:24.003690 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:34:24.003690 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:34:24.003690 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:34:24.004440 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:34:24.004440 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:34:24.004440 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:34:24.004440 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:34:24.004440 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:34:24.004440 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:34:24.004440 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:34:24.004440 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:34:24.004440 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:34:24.004440 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:34:24.004440 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:34:24.004440 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:34:24.004440 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 36/66] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 22:34:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:34:24.005542 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd78a0-9867-43d0-b7ff-8fc4bc3af392 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:24.005542 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd78a0-9867-43d0-b7ff-8fc4bc3af392 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:24.005779 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd78a0-9867-43d0-b7ff-8fc4bc3af392 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:24.006626 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-d9edb611-260b-4719-8b5b-6d2d232df9a6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:34:24.006626 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:34:24.006626 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:34:24.006626 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:34:24.006626 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:34:24.006626 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:34:24.006626 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:34:24.006626 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:34:24.006626 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:34:24.006626 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:34:24.006626 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:34:24.006626 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:34:24.006626 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:34:24.006626 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:34:24.006626 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:34:24.006626 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:34:24.007478 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:34:24.007478 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:34:24.007478 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:34:24.007478 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:34:24.007478 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:34:24.007478 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:34:24.007478 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:34:24.007478 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:34:24.007478 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:34:24.007478 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:34:24.007478 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:34:24.007478 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:34:24.007478 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:34:24.007478 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:34:24.007478 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:34:24.007478 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:34:24.008208 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:34:24.008208 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:34:24.008208 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:34:24.008208 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:34:24.008208 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:34:24.008208 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:34:24.008208 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:34:24.008208 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:34:24.008208 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:34:24.008208 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:34:24.008208 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:34:24.008208 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:34:24.008770 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 37/67] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:24.010572 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 31/68] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 21 22:34:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:34:24.012443 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfd78a0-9867-43d0-b7ff-8fc4bc3af392 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:24.013878 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1a06c-c058-4ad3-96f9-f8cae4091724 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:24.014360 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1a06c-c058-4ad3-96f9-f8cae4091724 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:24.015250 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1a06c-c058-4ad3-96f9-f8cae4091724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:24.038161 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:24.039145 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-fab1a06c-c058-4ad3-96f9-f8cae4091724 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:24.040443 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-fab1a06c-c058-4ad3-96f9-f8cae4091724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:24.119968 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 32/69] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:34:24.121688 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1a06c-c058-4ad3-96f9-f8cae4091724 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:24.163447 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfd78a0-9867-43d0-b7ff-8fc4bc3af392 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:24.207308 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:24.211373 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7541eda-aa64-4825-9ff7-85ab842f6da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:24.212059 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7541eda-aa64-4825-9ff7-85ab842f6da4 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:24.212348 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7541eda-aa64-4825-9ff7-85ab842f6da4 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:24.212632 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7541eda-aa64-4825-9ff7-85ab842f6da4 None admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:24.213400 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7541eda-aa64-4825-9ff7-85ab842f6da4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:24.213715 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7541eda-aa64-4825-9ff7-85ab842f6da4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:24.215191 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7541eda-aa64-4825-9ff7-85ab842f6da4 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:24.223914 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 38/70] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:34:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:24.224735 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7541eda-aa64-4825-9ff7-85ab842f6da4 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:24.242230 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1a06c-c058-4ad3-96f9-f8cae4091724 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:24.280343 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7da73ff-0201-4eaa-9115-be2da20741a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:24.281244 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da73ff-0201-4eaa-9115-be2da20741a9 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:24.281635 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da73ff-0201-4eaa-9115-be2da20741a9 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:24.281869 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da73ff-0201-4eaa-9115-be2da20741a9 None admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:24.282628 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7da73ff-0201-4eaa-9115-be2da20741a9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:24.283000 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7da73ff-0201-4eaa-9115-be2da20741a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:24.284457 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7da73ff-0201-4eaa-9115-be2da20741a9 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:24.293323 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 33/71] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:34:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:24.294509 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573c864-3333-4082-8c09-9ed3a973e161 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:24.294823 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573c864-3333-4082-8c09-9ed3a973e161 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:24.295109 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573c864-3333-4082-8c09-9ed3a973e161 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:24.296345 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 34/72] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:24.297042 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6573c864-3333-4082-8c09-9ed3a973e161 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:24.319691 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:24.338343 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7541eda-aa64-4825-9ff7-85ab842f6da4 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:24.373269 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7913a00f-bfbd-411e-830d-3235bcdecc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:24.374065 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7913a00f-bfbd-411e-830d-3235bcdecc59 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:24.374355 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7913a00f-bfbd-411e-830d-3235bcdecc59 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:24.374641 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7913a00f-bfbd-411e-830d-3235bcdecc59 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:24.384998 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7913a00f-bfbd-411e-830d-3235bcdecc59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:34:24.392652 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-7913a00f-bfbd-411e-830d-3235bcdecc59 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:34:24.392652 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:34:24.392652 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:34:24.392652 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:34:24.392652 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:34:24.392652 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:34:24.392652 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:34:24.392652 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:34:24.392652 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:34:24.392652 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:34:24.392652 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:34:24.392652 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:34:24.392652 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:34:24.392652 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:34:24.392652 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:34:24.392652 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:34:24.393568 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:34:24.393568 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:34:24.393568 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:34:24.393568 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:34:24.393568 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:34:24.393568 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:34:24.393568 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:34:24.393568 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:34:24.393568 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:34:24.393568 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:34:24.393568 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:34:24.393568 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:34:24.393568 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:34:24.393568 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:34:24.393568 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:34:24.393568 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:34:24.394352 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:34:24.394352 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:34:24.394352 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:34:24.394352 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:34:24.394352 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:34:24.394352 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:34:24.394352 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:34:24.394352 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:34:24.394352 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:34:24.394352 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:34:24.394352 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:34:24.394352 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:34:24.394352 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 39/73] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:34:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:34:24.395034 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e0829-de63-47eb-bcb7-9bab605ac9f5 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:24.395348 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e0829-de63-47eb-bcb7-9bab605ac9f5 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:24.395622 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e0829-de63-47eb-bcb7-9bab605ac9f5 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:24.396828 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 40/74] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:24.398718 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16e0829-de63-47eb-bcb7-9bab605ac9f5 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:24.402395 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6573c864-3333-4082-8c09-9ed3a973e161 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:24.431246 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:24.436716 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4098c4fa-9954-4506-b5e0-42a316588fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:24.437587 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098c4fa-9954-4506-b5e0-42a316588fc0 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:24.441252 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098c4fa-9954-4506-b5e0-42a316588fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:24.441252 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098c4fa-9954-4506-b5e0-42a316588fc0 None admin] PATH_INFO: `/v3/users/031023a71a014731ad7c782eb926f065` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:24.441252 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4098c4fa-9954-4506-b5e0-42a316588fc0 None admin] RBAC: Authorizing `identity:get_user(user_id=031023a71a014731ad7c782eb926f065)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:24.441252 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4098c4fa-9954-4506-b5e0-42a316588fc0 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:24.447257 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 35/75] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 22:34:24 2022] GET /identity/v3/users/031023a71a014731ad7c782eb926f065 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:24.447257 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cfaa61-a720-4ca9-88cf-326cd1df422b None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:24.447257 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cfaa61-a720-4ca9-88cf-326cd1df422b None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:24.447257 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cfaa61-a720-4ca9-88cf-326cd1df422b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:24.460925 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-77cfaa61-a720-4ca9-88cf-326cd1df422b None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:24.462109 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-77cfaa61-a720-4ca9-88cf-326cd1df422b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:24.505252 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16e0829-de63-47eb-bcb7-9bab605ac9f5 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:24.531073 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 36/76] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:34:24.531922 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cfaa61-a720-4ca9-88cf-326cd1df422b None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:24.541119 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daff51d-2b55-48a8-89a4-ca08482a770f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:24.542001 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daff51d-2b55-48a8-89a4-ca08482a770f None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:24.542798 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daff51d-2b55-48a8-89a4-ca08482a770f None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:24.543083 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daff51d-2b55-48a8-89a4-ca08482a770f None admin] PATH_INFO: `/v3/users/8e9d5ca05dea4d9c9dad9a3a31239f82` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:24.556111 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:24.560527 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daff51d-2b55-48a8-89a4-ca08482a770f None admin] RBAC: Authorizing `identity:get_user(user_id=8e9d5ca05dea4d9c9dad9a3a31239f82)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:24.562047 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daff51d-2b55-48a8-89a4-ca08482a770f None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:24.563725 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 41/77] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 22:34:24 2022] GET /identity/v3/users/8e9d5ca05dea4d9c9dad9a3a31239f82 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:24.568775 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e578f-8a98-4f32-acea-ea9f93422a01 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:24.569069 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e578f-8a98-4f32-acea-ea9f93422a01 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:24.569351 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e578f-8a98-4f32-acea-ea9f93422a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:24.584753 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-5f4e578f-8a98-4f32-acea-ea9f93422a01 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:24.585943 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-5f4e578f-8a98-4f32-acea-ea9f93422a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:24.641574 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cfaa61-a720-4ca9-88cf-326cd1df422b None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:24.651373 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 42/78] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:34:24.652698 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4e578f-8a98-4f32-acea-ea9f93422a01 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:24.671408 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:24.678086 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a52e89-e45d-4e72-baec-7012533817e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:24.681525 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a52e89-e45d-4e72-baec-7012533817e5 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:24.681525 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a52e89-e45d-4e72-baec-7012533817e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:24.681525 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a52e89-e45d-4e72-baec-7012533817e5 None admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:24.681525 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a52e89-e45d-4e72-baec-7012533817e5 None admin] RBAC: Adding query filter params (name=member) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:24.681525 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a52e89-e45d-4e72-baec-7012533817e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:24.681914 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a52e89-e45d-4e72-baec-7012533817e5 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:24.689791 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 37/79] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 22:34:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:24.691682 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a52e89-e45d-4e72-baec-7012533817e5 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:24.751934 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4e578f-8a98-4f32-acea-ea9f93422a01 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:24.777330 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:24.785087 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a72b5-c5f1-452d-92cf-6fc2a4e90280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:24.785759 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a72b5-c5f1-452d-92cf-6fc2a4e90280 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:24.790168 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a72b5-c5f1-452d-92cf-6fc2a4e90280 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:24.790168 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a72b5-c5f1-452d-92cf-6fc2a4e90280 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:24.790421 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a52e89-e45d-4e72-baec-7012533817e5 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:24.824321 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd7ce93-70a4-4cae-8dee-6a27f5535d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:24.825048 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd7ce93-70a4-4cae-8dee-6a27f5535d81 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:24.825334 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd7ce93-70a4-4cae-8dee-6a27f5535d81 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:24.825609 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd7ce93-70a4-4cae-8dee-6a27f5535d81 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:24.836512 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd7ce93-70a4-4cae-8dee-6a27f5535d81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:34:24.844696 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-ecd7ce93-70a4-4cae-8dee-6a27f5535d81 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:34:24.844696 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:34:24.844696 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:34:24.844696 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:34:24.844696 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:34:24.844696 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:34:24.844696 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:34:24.844696 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:34:24.844696 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:34:24.844696 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:34:24.844696 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:34:24.844696 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:34:24.844696 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:34:24.844696 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:34:24.844696 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:34:24.844696 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:34:24.845529 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:34:24.845529 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:34:24.845529 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:34:24.845529 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:34:24.845529 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:34:24.845529 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:34:24.845529 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:34:24.845529 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:34:24.845529 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:34:24.845529 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:34:24.845529 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:34:24.845529 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:34:24.845529 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:34:24.845529 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:34:24.845529 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:34:24.845529 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:34:24.846285 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:34:24.846285 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:34:24.846285 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:34:24.846285 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:34:24.846285 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:34:24.846285 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:34:24.846285 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:34:24.846285 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:34:24.846285 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:34:24.846285 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:34:24.846285 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:34:24.846285 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:34:24.846285 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 38/80] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:34:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:34:24.847417 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd7ce93-70a4-4cae-8dee-6a27f5535d81 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:24.857014 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3a72b5-c5f1-452d-92cf-6fc2a4e90280 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:24.858489 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3a72b5-c5f1-452d-92cf-6fc2a4e90280 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:24.901446 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:24.948495 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd7ce93-70a4-4cae-8dee-6a27f5535d81 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:24.955180 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 43/81] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:34:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:34:24.956012 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a72b5-c5f1-452d-92cf-6fc2a4e90280 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:24.983874 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab99cda-624e-49cf-93f7-8207d42ffad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:24.984334 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:24.984524 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab99cda-624e-49cf-93f7-8207d42ffad3 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:24.984813 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab99cda-624e-49cf-93f7-8207d42ffad3 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:24.985095 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab99cda-624e-49cf-93f7-8207d42ffad3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:25.052324 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab99cda-624e-49cf-93f7-8207d42ffad3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:25.053874 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab99cda-624e-49cf-93f7-8207d42ffad3 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:25.055957 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a72b5-c5f1-452d-92cf-6fc2a4e90280 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:25.090819 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed14aa32-859d-402a-8946-6e5ed02ea217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:25.091611 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14aa32-859d-402a-8946-6e5ed02ea217 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:25.091909 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14aa32-859d-402a-8946-6e5ed02ea217 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:25.092182 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14aa32-859d-402a-8946-6e5ed02ea217 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:25.093361 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed14aa32-859d-402a-8946-6e5ed02ea217 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:25.094829 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed14aa32-859d-402a-8946-6e5ed02ea217 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:25.096448 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 44/82] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:34:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:25.097206 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed14aa32-859d-402a-8946-6e5ed02ea217 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:25.108607 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:25.156060 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 39/83] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:34:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:34:25.156896 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab99cda-624e-49cf-93f7-8207d42ffad3 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:25.191503 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:25.197199 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed14aa32-859d-402a-8946-6e5ed02ea217 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:25.233474 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4eb1b-7e77-44dc-893c-78159502be60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:25.234358 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4eb1b-7e77-44dc-893c-78159502be60 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:25.234699 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4eb1b-7e77-44dc-893c-78159502be60 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:25.235009 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4eb1b-7e77-44dc-893c-78159502be60 None admin] PATH_INFO: `/v3/users/8e9d5ca05dea4d9c9dad9a3a31239f82` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:25.236215 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae4eb1b-7e77-44dc-893c-78159502be60 None admin] RBAC: Authorizing `identity:get_user(user_id=8e9d5ca05dea4d9c9dad9a3a31239f82)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:25.237674 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae4eb1b-7e77-44dc-893c-78159502be60 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:25.239352 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 45/84] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 22:34:25 2022] GET /identity/v3/users/8e9d5ca05dea4d9c9dad9a3a31239f82 => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:25.240120 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4eb1b-7e77-44dc-893c-78159502be60 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:25.265871 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab99cda-624e-49cf-93f7-8207d42ffad3 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:25.299005 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865678b0-e007-4924-9437-189db8ea8e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:25.299676 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865678b0-e007-4924-9437-189db8ea8e5e None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:25.299962 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865678b0-e007-4924-9437-189db8ea8e5e None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:25.300249 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865678b0-e007-4924-9437-189db8ea8e5e None admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:25.300991 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865678b0-e007-4924-9437-189db8ea8e5e None admin] RBAC: Adding query filter params (name=member) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:25.301306 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865678b0-e007-4924-9437-189db8ea8e5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:25.302693 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865678b0-e007-4924-9437-189db8ea8e5e None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:25.311283 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 40/85] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 22:34:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:25.312074 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865678b0-e007-4924-9437-189db8ea8e5e None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:25.327743 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:25.341882 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4eb1b-7e77-44dc-893c-78159502be60 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:25.375232 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aca1e64-93d4-4799-a966-f2c185927c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:25.376094 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca1e64-93d4-4799-a966-f2c185927c07 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:25.376377 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca1e64-93d4-4799-a966-f2c185927c07 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:25.377040 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca1e64-93d4-4799-a966-f2c185927c07 None admin] PATH_INFO: `/v3/projects/6edc20faecff41bf9b7ecc5ac4bc774e` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:25.377884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aca1e64-93d4-4799-a966-f2c185927c07 None admin] RBAC: Authorizing `identity:get_project(project_id=6edc20faecff41bf9b7ecc5ac4bc774e)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:25.379359 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aca1e64-93d4-4799-a966-f2c185927c07 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:25.381085 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 46/86] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:34:25 2022] GET /identity/v3/projects/6edc20faecff41bf9b7ecc5ac4bc774e => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:25.381839 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aca1e64-93d4-4799-a966-f2c185927c07 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:25.411329 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865678b0-e007-4924-9437-189db8ea8e5e None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:25.412631 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:25.444210 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766a1767-c689-46b7-82f3-a02b2463ab19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:25.444882 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a1767-c689-46b7-82f3-a02b2463ab19 None admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:25.445193 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a1767-c689-46b7-82f3-a02b2463ab19 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:25.445473 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a1767-c689-46b7-82f3-a02b2463ab19 None admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:25.446182 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766a1767-c689-46b7-82f3-a02b2463ab19 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:25.447666 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766a1767-c689-46b7-82f3-a02b2463ab19 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:25.483452 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aca1e64-93d4-4799-a966-f2c185927c07 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:25.516381 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dff3e29-5d82-4c60-9cba-00727415dc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:25.517242 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff3e29-5d82-4c60-9cba-00727415dc9c None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:25.517526 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff3e29-5d82-4c60-9cba-00727415dc9c None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:25.517803 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff3e29-5d82-4c60-9cba-00727415dc9c None admin] PATH_INFO: `/v3/projects/6edc20faecff41bf9b7ecc5ac4bc774e` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:25.519015 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dff3e29-5d82-4c60-9cba-00727415dc9c None admin] RBAC: Authorizing `identity:get_project(project_id=6edc20faecff41bf9b7ecc5ac4bc774e)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:25.520447 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dff3e29-5d82-4c60-9cba-00727415dc9c None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:25.522174 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 47/87] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:34:25 2022] GET /identity/v3/projects/6edc20faecff41bf9b7ecc5ac4bc774e => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:25.522925 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dff3e29-5d82-4c60-9cba-00727415dc9c None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:25.556162 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 41/88] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 21 22:34:25 2022] POST /identity/v3/users => generated 289 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:25.556995 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766a1767-c689-46b7-82f3-a02b2463ab19 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:25.640967 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:25.643278 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:25.645597 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dff3e29-5d82-4c60-9cba-00727415dc9c None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:25.680263 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766a1767-c689-46b7-82f3-a02b2463ab19 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:25.684883 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff6b97b-bca8-47be-b076-bd27bb6dae7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:25.685558 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff6b97b-bca8-47be-b076-bd27bb6dae7b None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:25.685849 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff6b97b-bca8-47be-b076-bd27bb6dae7b None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:25.686129 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff6b97b-bca8-47be-b076-bd27bb6dae7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:25.719500 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa92ffba-a4ab-40ce-aaf2-e792e445eb2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:25.720393 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa92ffba-a4ab-40ce-aaf2-e792e445eb2b None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:25.720686 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa92ffba-a4ab-40ce-aaf2-e792e445eb2b None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:25.721750 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa92ffba-a4ab-40ce-aaf2-e792e445eb2b None admin] PATH_INFO: `/v3/users/8e9d5ca05dea4d9c9dad9a3a31239f82` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:25.723568 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa92ffba-a4ab-40ce-aaf2-e792e445eb2b None admin] RBAC: Authorizing `identity:get_user(user_id=8e9d5ca05dea4d9c9dad9a3a31239f82)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:25.724505 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa92ffba-a4ab-40ce-aaf2-e792e445eb2b None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:25.726313 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 42/89] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 22:34:25 2022] GET /identity/v3/users/8e9d5ca05dea4d9c9dad9a3a31239f82 => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:25.729359 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa92ffba-a4ab-40ce-aaf2-e792e445eb2b None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:25.766985 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff6b97b-bca8-47be-b076-bd27bb6dae7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:25.768999 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff6b97b-bca8-47be-b076-bd27bb6dae7b None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:25.850696 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa92ffba-a4ab-40ce-aaf2-e792e445eb2b None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:25.891876 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9a669e-234f-4fdd-a331-04ab9e204ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:25.892464 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9a669e-234f-4fdd-a331-04ab9e204ee2 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:25.892750 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9a669e-234f-4fdd-a331-04ab9e204ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:25.893029 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9a669e-234f-4fdd-a331-04ab9e204ee2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:25.893870 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9a669e-234f-4fdd-a331-04ab9e204ee2 None admin] RBAC: Adding query filter params (role.id=d1ddee590a964ff1bbb4ee8716173021, scope.project.id=6edc20faecff41bf9b7ecc5ac4bc774e, user.id=031023a71a014731ad7c782eb926f065) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:25.894185 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9a669e-234f-4fdd-a331-04ab9e204ee2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:25.896609 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9a669e-234f-4fdd-a331-04ab9e204ee2 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:25.900095 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:25.906854 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 43/90] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 22:34:25 2022] GET /identity/v3/role_assignments?user.id=031023a71a014731ad7c782eb926f065&scope.project.id=6edc20faecff41bf9b7ecc5ac4bc774e&role.id=d1ddee590a964ff1bbb4ee8716173021 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:25.907870 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9a669e-234f-4fdd-a331-04ab9e204ee2 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:25.908504 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 48/91] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:34:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:34:25.909480 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff6b97b-bca8-47be-b076-bd27bb6dae7b None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:25.912945 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:26.055518 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff6b97b-bca8-47be-b076-bd27bb6dae7b None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:26.065888 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9a669e-234f-4fdd-a331-04ab9e204ee2 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:26.098571 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c835a76c-3182-4ebe-87a7-3076afcdc816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:26.099529 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c835a76c-3182-4ebe-87a7-3076afcdc816 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:26.099818 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c835a76c-3182-4ebe-87a7-3076afcdc816 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:26.100101 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c835a76c-3182-4ebe-87a7-3076afcdc816 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:26.142355 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01887c38-9b1a-4ff1-9be2-293d3c7c78d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:26.142976 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01887c38-9b1a-4ff1-9be2-293d3c7c78d8 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:26.143263 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01887c38-9b1a-4ff1-9be2-293d3c7c78d8 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:26.143544 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01887c38-9b1a-4ff1-9be2-293d3c7c78d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:26.144395 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01887c38-9b1a-4ff1-9be2-293d3c7c78d8 None admin] RBAC: Adding query filter params (role.id=b1849c34fc37410098b94ff687ab0055, scope.project.id=6edc20faecff41bf9b7ecc5ac4bc774e, user.id=8e9d5ca05dea4d9c9dad9a3a31239f82) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:26.144710 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01887c38-9b1a-4ff1-9be2-293d3c7c78d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:26.146159 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01887c38-9b1a-4ff1-9be2-293d3c7c78d8 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:26.158998 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 44/92] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 22:34:25 2022] GET /identity/v3/role_assignments?user.id=8e9d5ca05dea4d9c9dad9a3a31239f82&scope.project.id=6edc20faecff41bf9b7ecc5ac4bc774e&role.id=b1849c34fc37410098b94ff687ab0055 => generated 259 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:26.159863 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01887c38-9b1a-4ff1-9be2-293d3c7c78d8 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:26.252180 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:26.255047 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c835a76c-3182-4ebe-87a7-3076afcdc816 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:26.256546 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c835a76c-3182-4ebe-87a7-3076afcdc816 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:26.268763 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:26.306926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01887c38-9b1a-4ff1-9be2-293d3c7c78d8 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:26.343386 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d4023f-dbc6-4df1-bd1d-d1027e1b8e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:26.344264 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d4023f-dbc6-4df1-bd1d-d1027e1b8e1a None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:26.344550 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d4023f-dbc6-4df1-bd1d-d1027e1b8e1a None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:26.344835 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d4023f-dbc6-4df1-bd1d-d1027e1b8e1a None admin] PATH_INFO: `/v3/projects/6edc20faecff41bf9b7ecc5ac4bc774e` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:26.346032 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d4023f-dbc6-4df1-bd1d-d1027e1b8e1a None admin] RBAC: Authorizing `identity:get_project(project_id=6edc20faecff41bf9b7ecc5ac4bc774e)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:26.347968 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d4023f-dbc6-4df1-bd1d-d1027e1b8e1a None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:26.350175 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 45/93] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:34:26 2022] GET /identity/v3/projects/6edc20faecff41bf9b7ecc5ac4bc774e => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:26.356930 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d4023f-dbc6-4df1-bd1d-d1027e1b8e1a None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:26.465010 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 49/94] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:34:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:34:26.465670 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d4023f-dbc6-4df1-bd1d-d1027e1b8e1a None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:26.470605 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c835a76c-3182-4ebe-87a7-3076afcdc816 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:26.503348 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d5e7e5-302f-41e8-8660-a502f8e44f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:26.504018 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d5e7e5-302f-41e8-8660-a502f8e44f22 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:26.504306 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d5e7e5-302f-41e8-8660-a502f8e44f22 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:26.504590 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d5e7e5-302f-41e8-8660-a502f8e44f22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:26.505437 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d5e7e5-302f-41e8-8660-a502f8e44f22 None admin] RBAC: Adding query filter params (role.id=53d3bdbe9cdf4d35bc73854a33df8e45, scope.project.id=6edc20faecff41bf9b7ecc5ac4bc774e, user.id=8e9d5ca05dea4d9c9dad9a3a31239f82) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:26.505756 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d5e7e5-302f-41e8-8660-a502f8e44f22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:26.507325 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d5e7e5-302f-41e8-8660-a502f8e44f22 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:26.509278 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:26.513998 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 46/95] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 22:34:26 2022] GET /identity/v3/role_assignments?user.id=8e9d5ca05dea4d9c9dad9a3a31239f82&scope.project.id=6edc20faecff41bf9b7ecc5ac4bc774e&role.id=53d3bdbe9cdf4d35bc73854a33df8e45 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:34:26.628757 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:26.631819 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c835a76c-3182-4ebe-87a7-3076afcdc816 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:26.672935 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca1a689-b877-45f4-ae8c-3c45accc9bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:26.673811 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca1a689-b877-45f4-ae8c-3c45accc9bcf None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:26.674094 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca1a689-b877-45f4-ae8c-3c45accc9bcf None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:26.674370 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca1a689-b877-45f4-ae8c-3c45accc9bcf None admin] PATH_INFO: `/v3/projects/c07becf9369c4028b8884bbd96515aaf` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:26.675606 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca1a689-b877-45f4-ae8c-3c45accc9bcf None admin] RBAC: Authorizing `identity:get_project(project_id=c07becf9369c4028b8884bbd96515aaf)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:26.677048 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca1a689-b877-45f4-ae8c-3c45accc9bcf None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:26.679768 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 50/96] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:34:26 2022] GET /identity/v3/projects/c07becf9369c4028b8884bbd96515aaf => generated 312 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:34:26.686305 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d5e7e5-302f-41e8-8660-a502f8e44f22 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:26.817562 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d5e7e5-302f-41e8-8660-a502f8e44f22 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:26.863028 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06f7dae-7410-4f6c-ac88-11701b4f1e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:26.863756 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f7dae-7410-4f6c-ac88-11701b4f1e5d None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:26.864042 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f7dae-7410-4f6c-ac88-11701b4f1e5d None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:26.864321 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f7dae-7410-4f6c-ac88-11701b4f1e5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:26.865156 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06f7dae-7410-4f6c-ac88-11701b4f1e5d None admin] RBAC: Adding query filter params (role.id=53d3bdbe9cdf4d35bc73854a33df8e45, scope.project.id=c07becf9369c4028b8884bbd96515aaf, user.id=8e9d5ca05dea4d9c9dad9a3a31239f82) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:26.865474 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06f7dae-7410-4f6c-ac88-11701b4f1e5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:26.866938 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06f7dae-7410-4f6c-ac88-11701b4f1e5d None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:26.874920 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 47/97] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 22:34:26 2022] GET /identity/v3/role_assignments?user.id=8e9d5ca05dea4d9c9dad9a3a31239f82&scope.project.id=c07becf9369c4028b8884bbd96515aaf&role.id=53d3bdbe9cdf4d35bc73854a33df8e45 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:34:28.418320 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1a63e-4e22-4cd1-b0b7-c3f997801c9e None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:28.418886 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1a63e-4e22-4cd1-b0b7-c3f997801c9e None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:28.418886 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1a63e-4e22-4cd1-b0b7-c3f997801c9e None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:28.420621 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 51/98] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:28.436803 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c124aa-97ab-437b-80bc-95418a63dfd3 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:28.437350 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c124aa-97ab-437b-80bc-95418a63dfd3 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:28.437350 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c124aa-97ab-437b-80bc-95418a63dfd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:28.463776 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-c9c124aa-97ab-437b-80bc-95418a63dfd3 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:28.465075 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-c9c124aa-97ab-437b-80bc-95418a63dfd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:28.470894 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc0e2c6-71ee-4196-b866-04db82917dba None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:28.471479 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc0e2c6-71ee-4196-b866-04db82917dba None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:28.471955 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc0e2c6-71ee-4196-b866-04db82917dba None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:28.473542 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 52/99] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:34:28.483930 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ba1ed-0a21-4822-b8de-1357a244e9b7 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:28.484509 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ba1ed-0a21-4822-b8de-1357a244e9b7 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:28.485006 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ba1ed-0a21-4822-b8de-1357a244e9b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:28.519988 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-2b2ba1ed-0a21-4822-b8de-1357a244e9b7 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:28.521500 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-2b2ba1ed-0a21-4822-b8de-1357a244e9b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:28.577261 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:28.601383 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 48/100] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:34:28.644623 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c124aa-97ab-437b-80bc-95418a63dfd3 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:28.672776 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 53/101] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:34:28.715076 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2ba1ed-0a21-4822-b8de-1357a244e9b7 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:28.801273 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c124aa-97ab-437b-80bc-95418a63dfd3 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:28.879491 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0053644-0f12-4866-b99f-611b65f4a153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:28.880318 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0053644-0f12-4866-b99f-611b65f4a153 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:28.880613 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0053644-0f12-4866-b99f-611b65f4a153 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:28.880904 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0053644-0f12-4866-b99f-611b65f4a153 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:28.901531 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:28.905799 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2ba1ed-0a21-4822-b8de-1357a244e9b7 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:28.908094 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0053644-0f12-4866-b99f-611b65f4a153 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:34:28.924392 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-a0053644-0f12-4866-b99f-611b65f4a153 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:34:28.924392 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:34:28.924392 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:34:28.924392 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:34:28.924392 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:34:28.924392 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:34:28.924392 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:34:28.924392 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:34:28.924392 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:34:28.924392 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:34:28.924392 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:34:28.924392 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:34:28.924392 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:34:28.924392 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:34:28.924392 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:34:28.924392 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:34:28.925449 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:34:28.925449 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:34:28.925449 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:34:28.925449 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:34:28.925449 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:34:28.925449 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:34:28.925449 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:34:28.925449 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:34:28.925449 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:34:28.925449 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:34:28.925449 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:34:28.925449 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:34:28.925449 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:34:28.925449 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:34:28.925449 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:34:28.925449 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:34:28.926390 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:34:28.926390 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:34:28.926390 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:34:28.926390 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:34:28.926390 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:34:28.926390 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:34:28.926390 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:34:28.926390 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:34:28.926390 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:34:28.926390 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:34:28.926390 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:34:28.926390 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:34:28.926390 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 49/102] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:34:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:34:28.927115 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f501366c-a6d9-4001-8571-0626be2247f8 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:28.927115 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f501366c-a6d9-4001-8571-0626be2247f8 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:28.927400 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f501366c-a6d9-4001-8571-0626be2247f8 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:28.928819 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 50/103] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:28.935675 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f501366c-a6d9-4001-8571-0626be2247f8 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:28.946724 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6180fa-a1f6-44e7-b785-d0fb8daf6de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:28.947779 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6180fa-a1f6-44e7-b785-d0fb8daf6de5 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:28.948712 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6180fa-a1f6-44e7-b785-d0fb8daf6de5 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:28.949122 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6180fa-a1f6-44e7-b785-d0fb8daf6de5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:28.950488 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6180fa-a1f6-44e7-b785-d0fb8daf6de5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:28.952499 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6180fa-a1f6-44e7-b785-d0fb8daf6de5 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:28.954331 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 54/104] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:34:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:34:28.955751 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b96b8a-28dc-41a5-ba71-4bc1cb6c5f9d None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:28.956219 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b96b8a-28dc-41a5-ba71-4bc1cb6c5f9d None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:28.956737 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b96b8a-28dc-41a5-ba71-4bc1cb6c5f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:28.974591 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-d8b96b8a-28dc-41a5-ba71-4bc1cb6c5f9d None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:28.976143 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-d8b96b8a-28dc-41a5-ba71-4bc1cb6c5f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:29.051204 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 55/105] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:34:29.052296 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b96b8a-28dc-41a5-ba71-4bc1cb6c5f9d None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:29.055711 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:29.134289 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f501366c-a6d9-4001-8571-0626be2247f8 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:29.167157 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:29.175958 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b96b8a-28dc-41a5-ba71-4bc1cb6c5f9d None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:29.208428 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda21bd9-c8b3-47e7-afd0-a88e377d7b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:29.209166 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda21bd9-c8b3-47e7-afd0-a88e377d7b1e None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:29.209442 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda21bd9-c8b3-47e7-afd0-a88e377d7b1e None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:29.209778 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda21bd9-c8b3-47e7-afd0-a88e377d7b1e None admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:29.213090 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda21bd9-c8b3-47e7-afd0-a88e377d7b1e None admin] RBAC: Adding query filter params (name=reader) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:29.213090 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda21bd9-c8b3-47e7-afd0-a88e377d7b1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:29.214026 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda21bd9-c8b3-47e7-afd0-a88e377d7b1e None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:29.234988 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 51/106] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 22:34:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:29.235841 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda21bd9-c8b3-47e7-afd0-a88e377d7b1e None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:29.252104 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a296c79-3e13-40b0-a421-57b48ea39b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:29.253054 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a296c79-3e13-40b0-a421-57b48ea39b36 None admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:29.253487 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a296c79-3e13-40b0-a421-57b48ea39b36 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:29.253910 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a296c79-3e13-40b0-a421-57b48ea39b36 None admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:29.254765 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a296c79-3e13-40b0-a421-57b48ea39b36 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:29.256438 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a296c79-3e13-40b0-a421-57b48ea39b36 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:29.358651 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda21bd9-c8b3-47e7-afd0-a88e377d7b1e None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:29.373358 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 56/107] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jul 21 22:34:29 2022] POST /identity/v3/projects => generated 302 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:29.374439 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a296c79-3e13-40b0-a421-57b48ea39b36 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:29.383198 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:29.420202 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb001b47-b187-4dbc-bda1-3b770d911282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:29.421050 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb001b47-b187-4dbc-bda1-3b770d911282 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:29.421348 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb001b47-b187-4dbc-bda1-3b770d911282 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:29.421638 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb001b47-b187-4dbc-bda1-3b770d911282 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:29.444679 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb001b47-b187-4dbc-bda1-3b770d911282 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:34:29.460795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-bb001b47-b187-4dbc-bda1-3b770d911282 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:34:29.460795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:34:29.460795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:34:29.460795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:34:29.460795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:34:29.460795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:34:29.460795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:34:29.460795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:34:29.460795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:34:29.460795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:34:29.460795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:34:29.460795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:34:29.460795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:34:29.460795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:34:29.460795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:34:29.460795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:34:29.461699 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:34:29.461699 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:34:29.461699 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:34:29.461699 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:34:29.461699 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:34:29.461699 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:34:29.461699 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:34:29.461699 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:34:29.461699 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:34:29.461699 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:34:29.461699 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:34:29.461699 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:34:29.461699 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:34:29.461699 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:34:29.461699 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:34:29.461699 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:34:29.462449 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:34:29.462449 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:34:29.462449 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:34:29.462449 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:34:29.462449 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:34:29.462449 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:34:29.462449 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:34:29.462449 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:34:29.462449 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:34:29.462449 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:34:29.462449 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:34:29.462449 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:34:29.462449 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 52/108] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 22:34:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:34:29.471859 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb001b47-b187-4dbc-bda1-3b770d911282 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:29.491727 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a296c79-3e13-40b0-a421-57b48ea39b36 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:29.534526 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c5a07f-8c83-45d4-b29b-d9204ffd13db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:29.535948 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c5a07f-8c83-45d4-b29b-d9204ffd13db None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:29.536462 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c5a07f-8c83-45d4-b29b-d9204ffd13db None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:29.536902 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c5a07f-8c83-45d4-b29b-d9204ffd13db None admin] PATH_INFO: `/v3/users/a6860360a136470c9d8dcc9ac43849f9` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:29.553887 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c5a07f-8c83-45d4-b29b-d9204ffd13db None admin] RBAC: Authorizing `identity:get_user(user_id=a6860360a136470c9d8dcc9ac43849f9)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:29.556173 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c5a07f-8c83-45d4-b29b-d9204ffd13db None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:29.558372 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 57/109] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 22:34:29 2022] GET /identity/v3/users/a6860360a136470c9d8dcc9ac43849f9 => generated 289 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:29.564073 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c5a07f-8c83-45d4-b29b-d9204ffd13db None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:29.628550 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:29.680230 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb001b47-b187-4dbc-bda1-3b770d911282 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:29.723429 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c2852c-c73c-47cf-9be0-474eb4d018f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:29.724033 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2852c-c73c-47cf-9be0-474eb4d018f4 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:29.724319 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2852c-c73c-47cf-9be0-474eb4d018f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:29.724598 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2852c-c73c-47cf-9be0-474eb4d018f4 None admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:29.725349 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c2852c-c73c-47cf-9be0-474eb4d018f4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:29.725658 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c2852c-c73c-47cf-9be0-474eb4d018f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:29.727059 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c2852c-c73c-47cf-9be0-474eb4d018f4 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:29.736200 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 53/110] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:34:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:29.737482 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc21550-5aea-42a0-a115-b323972ad4fc None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:29.737770 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc21550-5aea-42a0-a115-b323972ad4fc None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:29.738050 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc21550-5aea-42a0-a115-b323972ad4fc None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:29.741145 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 54/111] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:29.743436 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9da6c2-c0e8-4e85-8c5d-6d09b3138330 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:29.743436 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9da6c2-c0e8-4e85-8c5d-6d09b3138330 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:29.743436 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9da6c2-c0e8-4e85-8c5d-6d09b3138330 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:29.748336 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 55/112] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:29.749115 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9da6c2-c0e8-4e85-8c5d-6d09b3138330 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:29.758230 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c5a07f-8c83-45d4-b29b-d9204ffd13db None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:29.795174 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:29.809581 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ff54a-1421-4517-aab9-80cbc4e523b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:29.810688 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ff54a-1421-4517-aab9-80cbc4e523b0 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:29.811221 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ff54a-1421-4517-aab9-80cbc4e523b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:29.811698 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ff54a-1421-4517-aab9-80cbc4e523b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:29.872498 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9da6c2-c0e8-4e85-8c5d-6d09b3138330 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:29.906874 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2ff54a-1421-4517-aab9-80cbc4e523b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:29.909014 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2ff54a-1421-4517-aab9-80cbc4e523b0 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:29.910399 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028b98c0-acc9-466f-9a6d-6dfc2ed1595a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:29.911411 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b98c0-acc9-466f-9a6d-6dfc2ed1595a None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:29.911788 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b98c0-acc9-466f-9a6d-6dfc2ed1595a None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:29.912158 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b98c0-acc9-466f-9a6d-6dfc2ed1595a None admin] PATH_INFO: `/v3/users/031023a71a014731ad7c782eb926f065` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:29.913855 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028b98c0-acc9-466f-9a6d-6dfc2ed1595a None admin] RBAC: Authorizing `identity:get_user(user_id=031023a71a014731ad7c782eb926f065)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:29.915063 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028b98c0-acc9-466f-9a6d-6dfc2ed1595a None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:29.922023 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 56/113] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 22:34:29 2022] GET /identity/v3/users/031023a71a014731ad7c782eb926f065 => generated 247 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:29.922023 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038b82c-94f4-44ea-b5c0-ddcb1913449e None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:29.922023 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038b82c-94f4-44ea-b5c0-ddcb1913449e None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:29.922023 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038b82c-94f4-44ea-b5c0-ddcb1913449e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:29.925209 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:29.953433 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-b038b82c-94f4-44ea-b5c0-ddcb1913449e None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:29.954878 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-b038b82c-94f4-44ea-b5c0-ddcb1913449e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:30.038048 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 58/114] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:34:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 22:34:30.039709 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb1393-063e-4e3b-be77-f9f4e87b8d9a None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:30.040140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb1393-063e-4e3b-be77-f9f4e87b8d9a None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:30.040581 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb1393-063e-4e3b-be77-f9f4e87b8d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:30.060770 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-fdcb1393-063e-4e3b-be77-f9f4e87b8d9a None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:30.062221 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-fdcb1393-063e-4e3b-be77-f9f4e87b8d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:30.080094 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 57/115] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:34:30.081022 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b038b82c-94f4-44ea-b5c0-ddcb1913449e None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:30.119226 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:30.134117 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 59/116] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:34:30.135851 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb1393-063e-4e3b-be77-f9f4e87b8d9a None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:30.172352 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:30.194985 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b038b82c-94f4-44ea-b5c0-ddcb1913449e None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:30.243862 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fde1387-e370-426e-ab33-63f30841f012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:30.244782 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde1387-e370-426e-ab33-63f30841f012 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:30.245080 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde1387-e370-426e-ab33-63f30841f012 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:30.245360 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde1387-e370-426e-ab33-63f30841f012 None admin] PATH_INFO: `/v3/projects/6edc20faecff41bf9b7ecc5ac4bc774e` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:30.248380 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fde1387-e370-426e-ab33-63f30841f012 None admin] RBAC: Authorizing `identity:get_project(project_id=6edc20faecff41bf9b7ecc5ac4bc774e)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:30.249841 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fde1387-e370-426e-ab33-63f30841f012 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:30.251579 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 58/117] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:34:30 2022] GET /identity/v3/projects/6edc20faecff41bf9b7ecc5ac4bc774e => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:30.252765 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17434bdd-52b6-4bce-b431-38e6b28ef920 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:30.253056 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17434bdd-52b6-4bce-b431-38e6b28ef920 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:30.253338 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17434bdd-52b6-4bce-b431-38e6b28ef920 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:30.254562 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 59/118] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:30.260207 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17434bdd-52b6-4bce-b431-38e6b28ef920 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:30.260644 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb1393-063e-4e3b-be77-f9f4e87b8d9a None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:30.306673 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b8284b-6cf4-4483-ad5e-627c5a6f0988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:30.307856 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b8284b-6cf4-4483-ad5e-627c5a6f0988 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:30.308380 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b8284b-6cf4-4483-ad5e-627c5a6f0988 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:30.308575 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b8284b-6cf4-4483-ad5e-627c5a6f0988 None admin] PATH_INFO: `/v3/projects/6edc20faecff41bf9b7ecc5ac4bc774e` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:30.309828 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b8284b-6cf4-4483-ad5e-627c5a6f0988 None admin] RBAC: Authorizing `identity:get_project(project_id=6edc20faecff41bf9b7ecc5ac4bc774e)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:30.311342 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b8284b-6cf4-4483-ad5e-627c5a6f0988 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:30.313045 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 60/119] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:34:30 2022] GET /identity/v3/projects/6edc20faecff41bf9b7ecc5ac4bc774e => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:30.313804 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b8284b-6cf4-4483-ad5e-627c5a6f0988 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:30.373263 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17434bdd-52b6-4bce-b431-38e6b28ef920 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:30.374846 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:30.408864 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2538fef-b10e-4b55-9502-69803c71278c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:30.409670 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2538fef-b10e-4b55-9502-69803c71278c None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:30.410011 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2538fef-b10e-4b55-9502-69803c71278c None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:30.410303 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2538fef-b10e-4b55-9502-69803c71278c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:30.421263 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2538fef-b10e-4b55-9502-69803c71278c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:34:30.427704 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b8284b-6cf4-4483-ad5e-627c5a6f0988 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:30.429258 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-d2538fef-b10e-4b55-9502-69803c71278c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:34:30.429258 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:34:30.429258 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:34:30.429258 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:34:30.429258 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:34:30.429258 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:34:30.429258 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:34:30.429258 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:34:30.429258 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:34:30.429258 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:34:30.429258 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:34:30.429258 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:34:30.429258 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:34:30.429258 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:34:30.429258 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:34:30.429258 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:34:30.430295 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:34:30.430295 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:34:30.430295 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:34:30.430295 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:34:30.430295 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:34:30.430295 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:34:30.430295 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:34:30.430295 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:34:30.430295 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:34:30.430295 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:34:30.430295 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:34:30.430295 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:34:30.430295 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:34:30.430295 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:34:30.430295 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:34:30.430295 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:34:30.431119 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:34:30.431119 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:34:30.431119 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:34:30.431119 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:34:30.431119 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:34:30.431119 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:34:30.431119 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:34:30.431119 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:34:30.431119 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:34:30.431119 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:34:30.431119 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:34:30.431119 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:34:30.431848 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:30.433207 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 60/120] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 21 22:34:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:34:30.434526 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ebbef9-1e22-44f2-a2fc-e34bb1a3b68d None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:30.434835 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ebbef9-1e22-44f2-a2fc-e34bb1a3b68d None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:30.435263 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ebbef9-1e22-44f2-a2fc-e34bb1a3b68d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:30.451875 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-55ebbef9-1e22-44f2-a2fc-e34bb1a3b68d None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:30.453055 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-55ebbef9-1e22-44f2-a2fc-e34bb1a3b68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:30.471546 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d855f8f-8a5b-46bd-bacb-0d999a4a6c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:30.472343 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d855f8f-8a5b-46bd-bacb-0d999a4a6c41 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:30.472785 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d855f8f-8a5b-46bd-bacb-0d999a4a6c41 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:30.473069 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d855f8f-8a5b-46bd-bacb-0d999a4a6c41 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:30.486599 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d855f8f-8a5b-46bd-bacb-0d999a4a6c41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:34:30.494270 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-3d855f8f-8a5b-46bd-bacb-0d999a4a6c41 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:34:30.494270 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:34:30.494270 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:34:30.494270 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:34:30.494270 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:34:30.494270 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:34:30.494270 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:34:30.494270 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:34:30.494270 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:34:30.494270 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:34:30.494270 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:34:30.494270 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:34:30.494270 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:34:30.494270 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:34:30.494270 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:34:30.494270 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:34:30.495710 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:34:30.495710 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:34:30.495710 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:34:30.495710 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:34:30.495710 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:34:30.495710 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:34:30.495710 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:34:30.495710 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:34:30.495710 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:34:30.495710 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:34:30.495710 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:34:30.495710 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:34:30.495710 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:34:30.495710 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:34:30.495710 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:34:30.495710 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:34:30.496582 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:34:30.496582 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:34:30.496582 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:34:30.496582 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:34:30.496582 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:34:30.496582 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:34:30.496582 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:34:30.496582 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:34:30.496582 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:34:30.496582 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:34:30.496582 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:34:30.496582 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:34:30.496582 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 61/121] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:34:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:34:30.496582 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d855f8f-8a5b-46bd-bacb-0d999a4a6c41 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:30.535279 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 61/122] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:34:30.536412 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ebbef9-1e22-44f2-a2fc-e34bb1a3b68d None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:30.625611 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d855f8f-8a5b-46bd-bacb-0d999a4a6c41 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:30.645128 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:30.654941 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ebbef9-1e22-44f2-a2fc-e34bb1a3b68d None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:30.660623 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d93b90-8b06-407d-bf87-e9cc415d69c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:30.661515 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d93b90-8b06-407d-bf87-e9cc415d69c1 None admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:30.661802 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d93b90-8b06-407d-bf87-e9cc415d69c1 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:30.662079 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d93b90-8b06-407d-bf87-e9cc415d69c1 None admin] PATH_INFO: `/v3/projects/6edc20faecff41bf9b7ecc5ac4bc774e/users/031023a71a014731ad7c782eb926f065/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:30.664268 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d93b90-8b06-407d-bf87-e9cc415d69c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=6edc20faecff41bf9b7ecc5ac4bc774e, user_id=031023a71a014731ad7c782eb926f065, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:30.665709 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d93b90-8b06-407d-bf87-e9cc415d69c1 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:30.690349 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6423bdae-8d2e-44fd-a5ec-527f775186c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:30.691097 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6423bdae-8d2e-44fd-a5ec-527f775186c1 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:30.691389 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6423bdae-8d2e-44fd-a5ec-527f775186c1 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:30.691864 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6423bdae-8d2e-44fd-a5ec-527f775186c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:30.692860 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6423bdae-8d2e-44fd-a5ec-527f775186c1 None admin] RBAC: Adding query filter params (role.id=6ea16f1a323a473e81c1b161134d7512, scope.project.id=6edc20faecff41bf9b7ecc5ac4bc774e, user.id=a6860360a136470c9d8dcc9ac43849f9) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:30.693541 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6423bdae-8d2e-44fd-a5ec-527f775186c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:30.695038 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6423bdae-8d2e-44fd-a5ec-527f775186c1 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:30.701109 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 62/123] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 22:34:30 2022] GET /identity/v3/role_assignments?user.id=a6860360a136470c9d8dcc9ac43849f9&scope.project.id=6edc20faecff41bf9b7ecc5ac4bc774e&role.id=6ea16f1a323a473e81c1b161134d7512 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:30.701876 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6423bdae-8d2e-44fd-a5ec-527f775186c1 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:30.755226 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 62/124] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 21 22:34:30 2022] PUT /identity/v3/projects/6edc20faecff41bf9b7ecc5ac4bc774e/users/031023a71a014731ad7c782eb926f065/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:30.758178 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d93b90-8b06-407d-bf87-e9cc415d69c1 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:30.800466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:30.898692 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6423bdae-8d2e-44fd-a5ec-527f775186c1 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:30.901277 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d93b90-8b06-407d-bf87-e9cc415d69c1 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:30.958973 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21988f36-acbc-4493-b344-ed5020eab5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:30.959899 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21988f36-acbc-4493-b344-ed5020eab5dc None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:30.961611 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21988f36-acbc-4493-b344-ed5020eab5dc None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:30.962814 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21988f36-acbc-4493-b344-ed5020eab5dc None admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:30.964252 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21988f36-acbc-4493-b344-ed5020eab5dc None admin] RBAC: Adding query filter params (name=member) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:30.965767 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21988f36-acbc-4493-b344-ed5020eab5dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:30.967911 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21988f36-acbc-4493-b344-ed5020eab5dc None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:30.971212 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0620ad-8d48-4857-a9b9-1085f6b1c522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:30.972114 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0620ad-8d48-4857-a9b9-1085f6b1c522 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:30.972537 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0620ad-8d48-4857-a9b9-1085f6b1c522 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:30.973846 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0620ad-8d48-4857-a9b9-1085f6b1c522 None admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:30.974761 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0620ad-8d48-4857-a9b9-1085f6b1c522 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:30.977408 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0620ad-8d48-4857-a9b9-1085f6b1c522 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:30.979507 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0620ad-8d48-4857-a9b9-1085f6b1c522 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:30.990834 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 63/125] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 22:34:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:30.990834 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21988f36-acbc-4493-b344-ed5020eab5dc None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:30.998848 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 63/126] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:34:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:31.002108 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0620ad-8d48-4857-a9b9-1085f6b1c522 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:31.023659 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:31.111288 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:31.126800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21988f36-acbc-4493-b344-ed5020eab5dc None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:31.129305 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0620ad-8d48-4857-a9b9-1085f6b1c522 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:31.161308 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20163c7e-b382-4f34-81c6-7c6e44f03781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:31.162041 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20163c7e-b382-4f34-81c6-7c6e44f03781 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:31.162329 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20163c7e-b382-4f34-81c6-7c6e44f03781 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:31.162614 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20163c7e-b382-4f34-81c6-7c6e44f03781 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:31.188281 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20163c7e-b382-4f34-81c6-7c6e44f03781 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:34:31.196060 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-20163c7e-b382-4f34-81c6-7c6e44f03781 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:34:31.196060 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:34:31.196060 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:34:31.196060 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:34:31.196060 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:34:31.196060 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:34:31.196060 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:34:31.196060 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:34:31.196060 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:34:31.196060 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:34:31.196060 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:34:31.196060 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:34:31.196060 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:34:31.196060 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:34:31.196060 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:34:31.196060 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:34:31.196992 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:34:31.196992 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:34:31.196992 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:34:31.196992 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:34:31.196992 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:34:31.196992 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:34:31.196992 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:34:31.196992 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:34:31.196992 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:34:31.196992 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:34:31.196992 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:34:31.196992 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:34:31.196992 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:34:31.196992 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:34:31.196992 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:34:31.196992 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:34:31.197752 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:34:31.197752 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:34:31.197752 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:34:31.197752 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:34:31.197752 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:34:31.197752 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:34:31.197752 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:34:31.197752 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:34:31.197752 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:34:31.197752 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:34:31.197752 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:34:31.197752 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:34:31.197752 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 64/127] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:34:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:34:31.198941 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20163c7e-b382-4f34-81c6-7c6e44f03781 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:31.206473 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc974d-c24c-40de-ab89-b744ba6b493d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:31.207384 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc974d-c24c-40de-ab89-b744ba6b493d None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:31.207669 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc974d-c24c-40de-ab89-b744ba6b493d None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:31.207951 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc974d-c24c-40de-ab89-b744ba6b493d None admin] PATH_INFO: `/v3/users/8e9d5ca05dea4d9c9dad9a3a31239f82` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:31.209231 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bc974d-c24c-40de-ab89-b744ba6b493d None admin] RBAC: Authorizing `identity:get_user(user_id=8e9d5ca05dea4d9c9dad9a3a31239f82)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:31.210685 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bc974d-c24c-40de-ab89-b744ba6b493d None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:31.212724 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 64/128] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 22:34:31 2022] GET /identity/v3/users/8e9d5ca05dea4d9c9dad9a3a31239f82 => generated 275 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:31.213351 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc974d-c24c-40de-ab89-b744ba6b493d None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:31.315098 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:31.331869 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc974d-c24c-40de-ab89-b744ba6b493d None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:31.369851 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb7d18-4c4f-49d1-b7ac-7f456219d17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:31.370747 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb7d18-4c4f-49d1-b7ac-7f456219d17a None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:31.374794 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb7d18-4c4f-49d1-b7ac-7f456219d17a None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:31.374794 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb7d18-4c4f-49d1-b7ac-7f456219d17a None admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:31.374794 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bb7d18-4c4f-49d1-b7ac-7f456219d17a None admin] RBAC: Adding query filter params (name=member) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:31.374794 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bb7d18-4c4f-49d1-b7ac-7f456219d17a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:31.374794 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bb7d18-4c4f-49d1-b7ac-7f456219d17a None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:31.385452 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 65/129] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 22:34:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:31.387946 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb7d18-4c4f-49d1-b7ac-7f456219d17a None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:31.474034 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20163c7e-b382-4f34-81c6-7c6e44f03781 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:31.521441 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb7d18-4c4f-49d1-b7ac-7f456219d17a None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:31.567569 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cc7312-8cbf-46a5-9e0c-3ddcf554413e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:31.568707 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cc7312-8cbf-46a5-9e0c-3ddcf554413e None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:31.568998 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cc7312-8cbf-46a5-9e0c-3ddcf554413e None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:31.569281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cc7312-8cbf-46a5-9e0c-3ddcf554413e None admin] PATH_INFO: `/v3/projects/6edc20faecff41bf9b7ecc5ac4bc774e` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:31.570509 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cc7312-8cbf-46a5-9e0c-3ddcf554413e None admin] RBAC: Authorizing `identity:get_project(project_id=6edc20faecff41bf9b7ecc5ac4bc774e)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:31.572693 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cc7312-8cbf-46a5-9e0c-3ddcf554413e None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:31.575168 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 66/130] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:34:31 2022] GET /identity/v3/projects/6edc20faecff41bf9b7ecc5ac4bc774e => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:31.576075 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cc7312-8cbf-46a5-9e0c-3ddcf554413e None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:31.596083 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed25669-b149-4151-a654-2311d888406c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:31.596984 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed25669-b149-4151-a654-2311d888406c None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:31.597268 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed25669-b149-4151-a654-2311d888406c None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:31.597541 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed25669-b149-4151-a654-2311d888406c None admin] PATH_INFO: `/v3/users/8e9d5ca05dea4d9c9dad9a3a31239f82` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:31.603535 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed25669-b149-4151-a654-2311d888406c None admin] RBAC: Authorizing `identity:get_user(user_id=8e9d5ca05dea4d9c9dad9a3a31239f82)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:31.605878 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed25669-b149-4151-a654-2311d888406c None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:31.607795 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 65/131] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 22:34:31 2022] GET /identity/v3/users/8e9d5ca05dea4d9c9dad9a3a31239f82 => generated 275 bytes in 410 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:31.608321 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed25669-b149-4151-a654-2311d888406c None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:31.615109 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:31.644318 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:31.699367 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cc7312-8cbf-46a5-9e0c-3ddcf554413e None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:31.761934 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6218c425-971c-4955-ae59-1369fae2983f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:31.763655 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6218c425-971c-4955-ae59-1369fae2983f None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:31.764156 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6218c425-971c-4955-ae59-1369fae2983f None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:31.765033 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6218c425-971c-4955-ae59-1369fae2983f None admin] PATH_INFO: `/v3/users/8e9d5ca05dea4d9c9dad9a3a31239f82` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:31.766715 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6218c425-971c-4955-ae59-1369fae2983f None admin] RBAC: Authorizing `identity:get_user(user_id=8e9d5ca05dea4d9c9dad9a3a31239f82)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:31.769263 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6218c425-971c-4955-ae59-1369fae2983f None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:31.771491 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 67/132] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 22:34:31 2022] GET /identity/v3/users/8e9d5ca05dea4d9c9dad9a3a31239f82 => generated 275 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:31.772923 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6218c425-971c-4955-ae59-1369fae2983f None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:31.790735 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed25669-b149-4151-a654-2311d888406c None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:31.828540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfc4d16-7564-421c-bffb-fbf7a08723b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:31.829403 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfc4d16-7564-421c-bffb-fbf7a08723b8 None admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:31.829697 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfc4d16-7564-421c-bffb-fbf7a08723b8 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:31.829980 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfc4d16-7564-421c-bffb-fbf7a08723b8 None admin] PATH_INFO: `/v3/projects/6edc20faecff41bf9b7ecc5ac4bc774e/users/8e9d5ca05dea4d9c9dad9a3a31239f82/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:31.832227 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfc4d16-7564-421c-bffb-fbf7a08723b8 None admin] RBAC: Authorizing `identity:create_grant(project_id=6edc20faecff41bf9b7ecc5ac4bc774e, user_id=8e9d5ca05dea4d9c9dad9a3a31239f82, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:31.833803 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfc4d16-7564-421c-bffb-fbf7a08723b8 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:31.869184 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:31.885954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6218c425-971c-4955-ae59-1369fae2983f None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:31.923276 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39511f1-80a1-4afd-9425-87c84765112d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:31.924168 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39511f1-80a1-4afd-9425-87c84765112d None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:31.924481 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39511f1-80a1-4afd-9425-87c84765112d None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:31.924729 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39511f1-80a1-4afd-9425-87c84765112d None admin] PATH_INFO: `/v3/projects/6edc20faecff41bf9b7ecc5ac4bc774e` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:31.926011 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39511f1-80a1-4afd-9425-87c84765112d None admin] RBAC: Authorizing `identity:get_project(project_id=6edc20faecff41bf9b7ecc5ac4bc774e)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:31.927571 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39511f1-80a1-4afd-9425-87c84765112d None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:31.933916 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 68/133] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:34:31 2022] GET /identity/v3/projects/6edc20faecff41bf9b7ecc5ac4bc774e => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:31.933916 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba60a30-83ed-414b-9fd2-218ac1146e0d None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:31.933916 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba60a30-83ed-414b-9fd2-218ac1146e0d None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:31.933916 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba60a30-83ed-414b-9fd2-218ac1146e0d None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:31.936328 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 69/134] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:31.936897 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba60a30-83ed-414b-9fd2-218ac1146e0d None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:32.000816 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 66/135] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 21 22:34:31 2022] PUT /identity/v3/projects/6edc20faecff41bf9b7ecc5ac4bc774e/users/8e9d5ca05dea4d9c9dad9a3a31239f82/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:32.002025 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfc4d16-7564-421c-bffb-fbf7a08723b8 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:32.113333 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba60a30-83ed-414b-9fd2-218ac1146e0d None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:32.143176 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:32.181925 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6340becf-d0b8-444b-84d9-bb2a1112b4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:32.183715 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6340becf-d0b8-444b-84d9-bb2a1112b4a9 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:32.184222 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6340becf-d0b8-444b-84d9-bb2a1112b4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:32.184717 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6340becf-d0b8-444b-84d9-bb2a1112b4a9 None admin] PATH_INFO: `/v3/projects/c07becf9369c4028b8884bbd96515aaf` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:32.187277 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6340becf-d0b8-444b-84d9-bb2a1112b4a9 None admin] RBAC: Authorizing `identity:get_project(project_id=c07becf9369c4028b8884bbd96515aaf)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:32.188533 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfc4d16-7564-421c-bffb-fbf7a08723b8 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:32.191641 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6340becf-d0b8-444b-84d9-bb2a1112b4a9 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:32.194334 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 70/136] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:34:31 2022] GET /identity/v3/projects/c07becf9369c4028b8884bbd96515aaf => generated 312 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:32.195957 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09682bf-f0f0-465b-bcbb-b7810005f800 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:32.197357 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09682bf-f0f0-465b-bcbb-b7810005f800 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:32.197856 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09682bf-f0f0-465b-bcbb-b7810005f800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:32.219011 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-f09682bf-f0f0-465b-bcbb-b7810005f800 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:32.221217 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-f09682bf-f0f0-465b-bcbb-b7810005f800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:32.237408 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a2db80-5f92-4694-a786-1449a4c3c167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:32.238309 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2db80-5f92-4694-a786-1449a4c3c167 None admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:32.238596 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2db80-5f92-4694-a786-1449a4c3c167 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:32.238904 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2db80-5f92-4694-a786-1449a4c3c167 None admin] PATH_INFO: `/v3/projects/6edc20faecff41bf9b7ecc5ac4bc774e/users/8e9d5ca05dea4d9c9dad9a3a31239f82/roles/b1849c34fc37410098b94ff687ab0055` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:32.241325 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a2db80-5f92-4694-a786-1449a4c3c167 None admin] RBAC: Authorizing `identity:create_grant(project_id=6edc20faecff41bf9b7ecc5ac4bc774e, user_id=8e9d5ca05dea4d9c9dad9a3a31239f82, role_id=b1849c34fc37410098b94ff687ab0055)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:32.242877 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a2db80-5f92-4694-a786-1449a4c3c167 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:32.243498 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:32.310495 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 67/137] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 21 22:34:32 2022] PUT /identity/v3/projects/6edc20faecff41bf9b7ecc5ac4bc774e/users/8e9d5ca05dea4d9c9dad9a3a31239f82/roles/b1849c34fc37410098b94ff687ab0055 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:32.313033 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a2db80-5f92-4694-a786-1449a4c3c167 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:32.321280 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 71/138] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:34:32.361463 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09682bf-f0f0-465b-bcbb-b7810005f800 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:32.517300 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a2db80-5f92-4694-a786-1449a4c3c167 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:32.582653 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09682bf-f0f0-465b-bcbb-b7810005f800 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:32.600351 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917fc4e8-4288-45ad-9f0e-a00b87701f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:32.601268 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917fc4e8-4288-45ad-9f0e-a00b87701f9e None admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:32.601556 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917fc4e8-4288-45ad-9f0e-a00b87701f9e None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:32.601838 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917fc4e8-4288-45ad-9f0e-a00b87701f9e None admin] PATH_INFO: `/v3/projects/c07becf9369c4028b8884bbd96515aaf/users/8e9d5ca05dea4d9c9dad9a3a31239f82/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:32.607207 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917fc4e8-4288-45ad-9f0e-a00b87701f9e None admin] RBAC: Authorizing `identity:create_grant(project_id=c07becf9369c4028b8884bbd96515aaf, user_id=8e9d5ca05dea4d9c9dad9a3a31239f82, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:32.608685 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917fc4e8-4288-45ad-9f0e-a00b87701f9e None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:32.611455 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:32.634483 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849fd2a0-754a-4558-9968-4ca5b311bfb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:32.635383 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849fd2a0-754a-4558-9968-4ca5b311bfb9 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:32.635701 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849fd2a0-754a-4558-9968-4ca5b311bfb9 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:32.636033 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849fd2a0-754a-4558-9968-4ca5b311bfb9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:32.638188 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849fd2a0-754a-4558-9968-4ca5b311bfb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:32.639876 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849fd2a0-754a-4558-9968-4ca5b311bfb9 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:32.641893 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 72/139] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:34:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:34:32.647801 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849fd2a0-754a-4558-9968-4ca5b311bfb9 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:32.675748 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:32.690704 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 68/140] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 21 22:34:32 2022] PUT /identity/v3/projects/c07becf9369c4028b8884bbd96515aaf/users/8e9d5ca05dea4d9c9dad9a3a31239f82/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:32.792278 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849fd2a0-754a-4558-9968-4ca5b311bfb9 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:32.835400 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db45f62-f747-47fd-9beb-f3894adf06bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:32.836098 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db45f62-f747-47fd-9beb-f3894adf06bf None admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:32.836389 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db45f62-f747-47fd-9beb-f3894adf06bf None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:32.836670 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db45f62-f747-47fd-9beb-f3894adf06bf None admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:32.837414 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db45f62-f747-47fd-9beb-f3894adf06bf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:32.839650 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db45f62-f747-47fd-9beb-f3894adf06bf None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:32.977219 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 73/141] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 21 22:34:32 2022] POST /identity/v3/users => generated 283 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:34:33.299597 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769816e-f6fb-4a43-ba4d-84ec08a9234b None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:33.299937 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769816e-f6fb-4a43-ba4d-84ec08a9234b None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:33.300174 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769816e-f6fb-4a43-ba4d-84ec08a9234b None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:33.301752 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 69/142] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:33.307005 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4f755-8b53-4c89-a7cd-a1c835936586 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:33.307333 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4f755-8b53-4c89-a7cd-a1c835936586 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:33.307616 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4f755-8b53-4c89-a7cd-a1c835936586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:33.310710 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ecb3ee-446b-4d56-b575-b5473c14d2dd None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:33.311042 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ecb3ee-446b-4d56-b575-b5473c14d2dd None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:33.311334 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ecb3ee-446b-4d56-b575-b5473c14d2dd None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:33.312814 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 70/143] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:34:33.319793 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c501a42-ff2d-418a-a4ff-26f704d591bc None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:33.320092 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c501a42-ff2d-418a-a4ff-26f704d591bc None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:33.320982 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c501a42-ff2d-418a-a4ff-26f704d591bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:33.331760 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-8aa4f755-8b53-4c89-a7cd-a1c835936586 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:33.332959 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-8aa4f755-8b53-4c89-a7cd-a1c835936586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:33.341150 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-8c501a42-ff2d-418a-a4ff-26f704d591bc None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:33.342356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-8c501a42-ff2d-418a-a4ff-26f704d591bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:33.452566 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 71/144] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:34:33.461818 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 74/145] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:34:33.476511 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c501a42-ff2d-418a-a4ff-26f704d591bc None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:33.495959 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa4f755-8b53-4c89-a7cd-a1c835936586 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:33.506613 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:33.636010 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c501a42-ff2d-418a-a4ff-26f704d591bc None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:33.670689 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:33.685369 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c465bb8a-13c2-4c8d-959d-f62037bfbc31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:33.686192 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465bb8a-13c2-4c8d-959d-f62037bfbc31 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:33.686547 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465bb8a-13c2-4c8d-959d-f62037bfbc31 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:33.690893 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465bb8a-13c2-4c8d-959d-f62037bfbc31 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:33.695579 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa4f755-8b53-4c89-a7cd-a1c835936586 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:33.708598 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c465bb8a-13c2-4c8d-959d-f62037bfbc31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:34:33.720073 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-c465bb8a-13c2-4c8d-959d-f62037bfbc31 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:34:33.720073 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:34:33.720073 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:34:33.720073 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:34:33.720073 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:34:33.720073 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:34:33.720073 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:34:33.720073 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:34:33.720073 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:34:33.720073 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:34:33.720073 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:34:33.720073 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:34:33.720073 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:34:33.720073 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:34:33.720073 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:34:33.720073 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:34:33.721005 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:34:33.721005 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:34:33.721005 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:34:33.721005 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:34:33.721005 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:34:33.721005 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:34:33.721005 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:34:33.721005 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:34:33.721005 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:34:33.721005 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:34:33.721005 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:34:33.721005 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:34:33.721005 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:34:33.721005 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:34:33.721005 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:34:33.721005 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:34:33.721788 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:34:33.721788 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:34:33.721788 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:34:33.721788 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:34:33.721788 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:34:33.721788 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:34:33.721788 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:34:33.721788 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:34:33.721788 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:34:33.721788 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:34:33.721788 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:34:33.721788 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:34:33.721788 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 72/146] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 22:34:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:34:33.729016 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c465bb8a-13c2-4c8d-959d-f62037bfbc31 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:33.744439 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acb9f92-0bfe-42c4-a41d-d0762802e3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:33.745788 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb9f92-0bfe-42c4-a41d-d0762802e3a5 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:33.746117 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb9f92-0bfe-42c4-a41d-d0762802e3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:33.746343 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb9f92-0bfe-42c4-a41d-d0762802e3a5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:33.767939 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acb9f92-0bfe-42c4-a41d-d0762802e3a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:34:33.775720 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-6acb9f92-0bfe-42c4-a41d-d0762802e3a5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:34:33.775720 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:34:33.775720 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:34:33.775720 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:34:33.775720 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:34:33.775720 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:34:33.775720 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:34:33.775720 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:34:33.775720 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:34:33.775720 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:34:33.775720 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:34:33.775720 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:34:33.775720 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:34:33.775720 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:34:33.775720 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:34:33.775720 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:34:33.776645 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:34:33.776645 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:34:33.776645 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:34:33.776645 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:34:33.776645 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:34:33.776645 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:34:33.776645 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:34:33.776645 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:34:33.776645 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:34:33.776645 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:34:33.776645 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:34:33.776645 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:34:33.776645 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:34:33.776645 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:34:33.776645 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:34:33.776645 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:34:33.777450 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:34:33.777450 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:34:33.777450 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:34:33.777450 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:34:33.777450 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:34:33.777450 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:34:33.777450 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:34:33.777450 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:34:33.777450 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:34:33.777450 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:34:33.777450 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:34:33.777450 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:34:33.777450 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 75/147] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:34:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:34:33.784412 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acb9f92-0bfe-42c4-a41d-d0762802e3a5 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:33.857691 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:33.896020 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c465bb8a-13c2-4c8d-959d-f62037bfbc31 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:33.900623 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acb9f92-0bfe-42c4-a41d-d0762802e3a5 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:33.943501 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d1f73b-1232-4174-a232-1c8e35dd6b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:33.944200 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1f73b-1232-4174-a232-1c8e35dd6b71 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:33.944490 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1f73b-1232-4174-a232-1c8e35dd6b71 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:33.944772 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1f73b-1232-4174-a232-1c8e35dd6b71 None admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:33.945560 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d1f73b-1232-4174-a232-1c8e35dd6b71 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:33.945836 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d1f73b-1232-4174-a232-1c8e35dd6b71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:33.947366 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d1f73b-1232-4174-a232-1c8e35dd6b71 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:33.957563 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 73/148] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:34:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:33.959495 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7762001-1641-4aa5-affb-af2dcebb4ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:33.960184 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7762001-1641-4aa5-affb-af2dcebb4ac9 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:33.960470 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7762001-1641-4aa5-affb-af2dcebb4ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:33.960750 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7762001-1641-4aa5-affb-af2dcebb4ac9 None admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:33.961519 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7762001-1641-4aa5-affb-af2dcebb4ac9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:33.961828 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7762001-1641-4aa5-affb-af2dcebb4ac9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:33.963352 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7762001-1641-4aa5-affb-af2dcebb4ac9 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:33.965014 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d1f73b-1232-4174-a232-1c8e35dd6b71 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:33.972677 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 76/149] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 22:34:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:34:33.979416 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7762001-1641-4aa5-affb-af2dcebb4ac9 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:34.069543 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:34.148913 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7762001-1641-4aa5-affb-af2dcebb4ac9 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:34.174780 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d1f73b-1232-4174-a232-1c8e35dd6b71 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:34.191559 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:34.211503 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a8b4d-6e28-4aae-9e9f-4bd10c6de4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:34.212404 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a8b4d-6e28-4aae-9e9f-4bd10c6de4c4 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:34.212690 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a8b4d-6e28-4aae-9e9f-4bd10c6de4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:34.212970 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a8b4d-6e28-4aae-9e9f-4bd10c6de4c4 None admin] PATH_INFO: `/v3/users/a6860360a136470c9d8dcc9ac43849f9` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:34.214906 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8a8b4d-6e28-4aae-9e9f-4bd10c6de4c4 None admin] RBAC: Authorizing `identity:get_user(user_id=a6860360a136470c9d8dcc9ac43849f9)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:34.216362 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8a8b4d-6e28-4aae-9e9f-4bd10c6de4c4 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:34.218015 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 77/150] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 22:34:33 2022] GET /identity/v3/users/a6860360a136470c9d8dcc9ac43849f9 => generated 289 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:34.223386 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a8b4d-6e28-4aae-9e9f-4bd10c6de4c4 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:34.251568 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04829961-c3bb-463c-b1ce-9932645ea2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:34.252420 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04829961-c3bb-463c-b1ce-9932645ea2a4 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:34.252745 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04829961-c3bb-463c-b1ce-9932645ea2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:34.252998 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04829961-c3bb-463c-b1ce-9932645ea2a4 None admin] PATH_INFO: `/v3/users/031023a71a014731ad7c782eb926f065` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:34.254278 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04829961-c3bb-463c-b1ce-9932645ea2a4 None admin] RBAC: Authorizing `identity:get_user(user_id=031023a71a014731ad7c782eb926f065)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:34.255775 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04829961-c3bb-463c-b1ce-9932645ea2a4 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:34.257425 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 74/151] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 22:34:33 2022] GET /identity/v3/users/031023a71a014731ad7c782eb926f065 => generated 247 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:34.267779 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04829961-c3bb-463c-b1ce-9932645ea2a4 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:34.415596 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04829961-c3bb-463c-b1ce-9932645ea2a4 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:34.427926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a8b4d-6e28-4aae-9e9f-4bd10c6de4c4 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:34.436030 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:34.462284 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558fe4d-3eb6-420e-9dfb-97c6d4d633ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:34.462981 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558fe4d-3eb6-420e-9dfb-97c6d4d633ae None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:34.463274 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558fe4d-3eb6-420e-9dfb-97c6d4d633ae None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:34.463589 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558fe4d-3eb6-420e-9dfb-97c6d4d633ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:34.523116 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678b82c6-7691-4fa7-b86e-a6e22b548d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:34.523997 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b82c6-7691-4fa7-b86e-a6e22b548d8b None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:34.524287 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b82c6-7691-4fa7-b86e-a6e22b548d8b None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:34.524566 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b82c6-7691-4fa7-b86e-a6e22b548d8b None admin] PATH_INFO: `/v3/projects/6edc20faecff41bf9b7ecc5ac4bc774e` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:34.525835 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678b82c6-7691-4fa7-b86e-a6e22b548d8b None admin] RBAC: Authorizing `identity:get_project(project_id=6edc20faecff41bf9b7ecc5ac4bc774e)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:34.533328 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678b82c6-7691-4fa7-b86e-a6e22b548d8b None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:34.535222 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 78/152] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:34:34 2022] GET /identity/v3/projects/6edc20faecff41bf9b7ecc5ac4bc774e => generated 298 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:34.541043 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678b82c6-7691-4fa7-b86e-a6e22b548d8b None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:34.596144 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4558fe4d-3eb6-420e-9dfb-97c6d4d633ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:34.597632 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4558fe4d-3eb6-420e-9dfb-97c6d4d633ae None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:34.659847 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:34.716374 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678b82c6-7691-4fa7-b86e-a6e22b548d8b None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:34.769354 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af911b-3e42-4441-9c1b-5390248f87be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:34.770261 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af911b-3e42-4441-9c1b-5390248f87be None admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:34.770546 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af911b-3e42-4441-9c1b-5390248f87be None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:34.770855 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af911b-3e42-4441-9c1b-5390248f87be None admin] PATH_INFO: `/v3/projects/6edc20faecff41bf9b7ecc5ac4bc774e/users/a6860360a136470c9d8dcc9ac43849f9/roles/6ea16f1a323a473e81c1b161134d7512` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:34.776157 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9af911b-3e42-4441-9c1b-5390248f87be None admin] RBAC: Authorizing `identity:create_grant(project_id=6edc20faecff41bf9b7ecc5ac4bc774e, user_id=a6860360a136470c9d8dcc9ac43849f9, role_id=6ea16f1a323a473e81c1b161134d7512)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:34.777628 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9af911b-3e42-4441-9c1b-5390248f87be None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:34.816267 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 75/153] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:34:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:34:34.822509 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558fe4d-3eb6-420e-9dfb-97c6d4d633ae None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:34.838133 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:34.885263 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 79/154] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 21 22:34:34 2022] PUT /identity/v3/projects/6edc20faecff41bf9b7ecc5ac4bc774e/users/a6860360a136470c9d8dcc9ac43849f9/roles/6ea16f1a323a473e81c1b161134d7512 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:34:34.951789 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558fe4d-3eb6-420e-9dfb-97c6d4d633ae None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:34.998711 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97385f85-d109-4fc3-a909-ccc8c279852b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:34.999568 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97385f85-d109-4fc3-a909-ccc8c279852b None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:34.999855 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97385f85-d109-4fc3-a909-ccc8c279852b None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:35.000134 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97385f85-d109-4fc3-a909-ccc8c279852b None admin] PATH_INFO: `/v3/projects/6edc20faecff41bf9b7ecc5ac4bc774e` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:35.001501 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97385f85-d109-4fc3-a909-ccc8c279852b None admin] RBAC: Authorizing `identity:get_project(project_id=6edc20faecff41bf9b7ecc5ac4bc774e)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:35.003022 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97385f85-d109-4fc3-a909-ccc8c279852b None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:35.004980 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 76/155] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:34:34 2022] GET /identity/v3/projects/6edc20faecff41bf9b7ecc5ac4bc774e => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:34:35.011889 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af911b-3e42-4441-9c1b-5390248f87be None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:35.218846 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af911b-3e42-4441-9c1b-5390248f87be None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:35.256732 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:35.269776 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26258315-f03a-455e-9549-a66b624ef0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:35.270451 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26258315-f03a-455e-9549-a66b624ef0e9 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:35.270739 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26258315-f03a-455e-9549-a66b624ef0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:35.271063 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26258315-f03a-455e-9549-a66b624ef0e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:35.271919 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26258315-f03a-455e-9549-a66b624ef0e9 None admin] RBAC: Adding query filter params (role.id=d1ddee590a964ff1bbb4ee8716173021, scope.project.id=6edc20faecff41bf9b7ecc5ac4bc774e, user.id=031023a71a014731ad7c782eb926f065) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:35.272298 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26258315-f03a-455e-9549-a66b624ef0e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:35.273771 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26258315-f03a-455e-9549-a66b624ef0e9 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:35.283862 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 80/156] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 22:34:35 2022] GET /identity/v3/role_assignments?user.id=031023a71a014731ad7c782eb926f065&scope.project.id=6edc20faecff41bf9b7ecc5ac4bc774e&role.id=d1ddee590a964ff1bbb4ee8716173021 => generated 608 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:35.372229 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a842c4-51b4-4985-9025-0c2abe172990 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:35.372548 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a842c4-51b4-4985-9025-0c2abe172990 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:35.372799 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a842c4-51b4-4985-9025-0c2abe172990 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:35.374634 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 77/157] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:35.389139 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57241ae7-a231-4c2f-af34-530d0cd3cd2f None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:35.389456 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57241ae7-a231-4c2f-af34-530d0cd3cd2f None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:35.389713 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57241ae7-a231-4c2f-af34-530d0cd3cd2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:35.416216 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-57241ae7-a231-4c2f-af34-530d0cd3cd2f None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:35.417987 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-57241ae7-a231-4c2f-af34-530d0cd3cd2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:35.505835 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 81/158] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:34:35.556834 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a842c4-51b4-4985-9025-0c2abe172990 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:35.666181 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:35.692630 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a842c4-51b4-4985-9025-0c2abe172990 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:35.738574 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018f626e-72ee-4ed9-9180-b73962ba2191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:35.739443 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f626e-72ee-4ed9-9180-b73962ba2191 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:35.739728 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f626e-72ee-4ed9-9180-b73962ba2191 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:35.740007 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f626e-72ee-4ed9-9180-b73962ba2191 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:35.756363 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018f626e-72ee-4ed9-9180-b73962ba2191 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:34:35.764622 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-018f626e-72ee-4ed9-9180-b73962ba2191 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:34:35.764622 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:34:35.764622 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:34:35.764622 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:34:35.764622 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:34:35.764622 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:34:35.764622 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:34:35.764622 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:34:35.764622 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:34:35.764622 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:34:35.764622 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:34:35.764622 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:34:35.764622 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:34:35.764622 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:34:35.764622 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:34:35.764622 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:34:35.765492 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:34:35.765492 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:34:35.765492 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:34:35.765492 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:34:35.765492 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:34:35.765492 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:34:35.765492 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:34:35.765492 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:34:35.765492 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:34:35.765492 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:34:35.765492 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:34:35.765492 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:34:35.765492 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:34:35.765492 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:34:35.765492 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:34:35.765492 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:34:35.766344 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:34:35.766344 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:34:35.766344 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:34:35.766344 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:34:35.766344 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:34:35.766344 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:34:35.766344 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:34:35.766344 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:34:35.766344 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:34:35.766344 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:34:35.766344 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:34:35.766344 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:34:35.766344 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 78/159] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:34:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:34:35.772627 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57241ae7-a231-4c2f-af34-530d0cd3cd2f None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:35.893634 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57241ae7-a231-4c2f-af34-530d0cd3cd2f None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:35.916434 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:35.937375 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd015781-8e88-4ce2-80dc-299fc44e670b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:35.938074 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd015781-8e88-4ce2-80dc-299fc44e670b None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:35.938358 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd015781-8e88-4ce2-80dc-299fc44e670b None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:35.938633 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd015781-8e88-4ce2-80dc-299fc44e670b None admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:35.939425 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd015781-8e88-4ce2-80dc-299fc44e670b None admin] RBAC: Adding query filter params (name=member) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:35.939772 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd015781-8e88-4ce2-80dc-299fc44e670b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:35.941135 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd015781-8e88-4ce2-80dc-299fc44e670b None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:35.955980 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 82/160] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 22:34:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:35.961752 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018f626e-72ee-4ed9-9180-b73962ba2191 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:36.081005 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018f626e-72ee-4ed9-9180-b73962ba2191 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:36.122871 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b6a89c-f9a6-4cbb-bb2b-303f48170ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:36.123770 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6a89c-f9a6-4cbb-bb2b-303f48170ca1 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:36.124054 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6a89c-f9a6-4cbb-bb2b-303f48170ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:36.124333 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6a89c-f9a6-4cbb-bb2b-303f48170ca1 None admin] PATH_INFO: `/v3/users/8e9d5ca05dea4d9c9dad9a3a31239f82` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:36.125586 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b6a89c-f9a6-4cbb-bb2b-303f48170ca1 None admin] RBAC: Authorizing `identity:get_user(user_id=8e9d5ca05dea4d9c9dad9a3a31239f82)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:36.127072 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b6a89c-f9a6-4cbb-bb2b-303f48170ca1 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:36.128729 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 79/161] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 22:34:35 2022] GET /identity/v3/users/8e9d5ca05dea4d9c9dad9a3a31239f82 => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:34:36.134369 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd015781-8e88-4ce2-80dc-299fc44e670b None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:36.278120 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ee119-b933-433f-9db6-6c835c0e61ac None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:36.278386 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ee119-b933-433f-9db6-6c835c0e61ac None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:36.278710 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ee119-b933-433f-9db6-6c835c0e61ac None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:36.280191 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 80/162] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:34:36.286992 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21054081-0efe-4e5d-9ef1-0019ea7e4b34 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:36.287284 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21054081-0efe-4e5d-9ef1-0019ea7e4b34 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:36.287569 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21054081-0efe-4e5d-9ef1-0019ea7e4b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:36.313602 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-21054081-0efe-4e5d-9ef1-0019ea7e4b34 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:36.314830 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-21054081-0efe-4e5d-9ef1-0019ea7e4b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:36.331417 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:36.348201 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd015781-8e88-4ce2-80dc-299fc44e670b None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:36.425409 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04527fd7-4601-41c7-b7d7-ca33a6130405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:36.426071 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04527fd7-4601-41c7-b7d7-ca33a6130405 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:36.426357 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04527fd7-4601-41c7-b7d7-ca33a6130405 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:36.426633 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04527fd7-4601-41c7-b7d7-ca33a6130405 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:36.464237 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 81/163] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:34:36.509639 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21054081-0efe-4e5d-9ef1-0019ea7e4b34 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:36.533583 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:36.581640 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04527fd7-4601-41c7-b7d7-ca33a6130405 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:36.583170 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04527fd7-4601-41c7-b7d7-ca33a6130405 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:36.692615 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21054081-0efe-4e5d-9ef1-0019ea7e4b34 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:36.740896 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73ff5f8-2ea0-44cb-bda2-3938cf715379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:36.741718 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73ff5f8-2ea0-44cb-bda2-3938cf715379 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:36.742005 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73ff5f8-2ea0-44cb-bda2-3938cf715379 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:36.742285 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73ff5f8-2ea0-44cb-bda2-3938cf715379 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:36.754795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 83/164] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:34:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 621 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:34:36.761203 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7d51f-6dc0-40e8-b036-2ae1875a75ec None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:36.761491 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7d51f-6dc0-40e8-b036-2ae1875a75ec None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:36.761773 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7d51f-6dc0-40e8-b036-2ae1875a75ec None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:36.763174 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 84/165] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:36 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:36.764134 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7d51f-6dc0-40e8-b036-2ae1875a75ec None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:36.767380 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:36.776520 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73ff5f8-2ea0-44cb-bda2-3938cf715379 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:34:36.794116 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-f73ff5f8-2ea0-44cb-bda2-3938cf715379 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:34:36.794116 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:34:36.794116 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:34:36.794116 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:34:36.794116 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:34:36.794116 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:34:36.794116 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:34:36.794116 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:34:36.794116 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:34:36.794116 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:34:36.794116 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:34:36.794116 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:34:36.794116 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:34:36.794116 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:34:36.794116 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:34:36.794116 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:34:36.795061 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:34:36.795061 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:34:36.795061 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:34:36.795061 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:34:36.795061 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:34:36.795061 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:34:36.795061 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:34:36.795061 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:34:36.795061 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:34:36.795061 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:34:36.795061 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:34:36.795061 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:34:36.795061 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:34:36.795061 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:34:36.795061 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:34:36.795061 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:34:36.795950 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:34:36.795950 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:34:36.795950 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:34:36.795950 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:34:36.795950 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:34:36.795950 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:34:36.795950 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:34:36.795950 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:34:36.795950 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:34:36.795950 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:34:36.795950 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:34:36.795950 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:34:36.795950 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 82/166] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:34:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:34:36.796650 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4f210-dfc3-476f-bdd4-4bd75d4b66f5 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:36.797439 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4f210-dfc3-476f-bdd4-4bd75d4b66f5 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:36.797439 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4f210-dfc3-476f-bdd4-4bd75d4b66f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:36.831802 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-d3b4f210-dfc3-476f-bdd4-4bd75d4b66f5 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:36.833015 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-d3b4f210-dfc3-476f-bdd4-4bd75d4b66f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:36.933466 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:36.952817 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 83/167] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:34:36.953770 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b4f210-dfc3-476f-bdd4-4bd75d4b66f5 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:37.013673 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7d51f-6dc0-40e8-b036-2ae1875a75ec None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:37.087496 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4657acd-e3d5-40ea-9ca3-39dd52a650cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:37.088416 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4657acd-e3d5-40ea-9ca3-39dd52a650cd None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:37.088703 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4657acd-e3d5-40ea-9ca3-39dd52a650cd None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:37.088984 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4657acd-e3d5-40ea-9ca3-39dd52a650cd None admin] PATH_INFO: `/v3/projects/6edc20faecff41bf9b7ecc5ac4bc774e` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:37.090212 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4657acd-e3d5-40ea-9ca3-39dd52a650cd None admin] RBAC: Authorizing `identity:get_project(project_id=6edc20faecff41bf9b7ecc5ac4bc774e)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:37.091702 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4657acd-e3d5-40ea-9ca3-39dd52a650cd None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:37.093574 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 85/168] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:34:36 2022] GET /identity/v3/projects/6edc20faecff41bf9b7ecc5ac4bc774e => generated 298 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:37.094169 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4657acd-e3d5-40ea-9ca3-39dd52a650cd None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:37.123246 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b4f210-dfc3-476f-bdd4-4bd75d4b66f5 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:37.183126 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:37.192127 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d60f70-6349-4ecf-9f15-9710408807fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:37.192863 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d60f70-6349-4ecf-9f15-9710408807fa None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:37.193157 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d60f70-6349-4ecf-9f15-9710408807fa None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:37.193438 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d60f70-6349-4ecf-9f15-9710408807fa None admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:37.194200 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d60f70-6349-4ecf-9f15-9710408807fa None admin] RBAC: Adding query filter params (name=member) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:37.194511 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d60f70-6349-4ecf-9f15-9710408807fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:37.195993 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d60f70-6349-4ecf-9f15-9710408807fa None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:37.211957 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 84/169] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 22:34:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:37.212817 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d60f70-6349-4ecf-9f15-9710408807fa None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:37.277398 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4657acd-e3d5-40ea-9ca3-39dd52a650cd None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:37.313196 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf9162c-ac2f-43eb-9a15-9e99678a929f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:37.314008 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9162c-ac2f-43eb-9a15-9e99678a929f None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:37.314294 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9162c-ac2f-43eb-9a15-9e99678a929f None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:37.314580 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9162c-ac2f-43eb-9a15-9e99678a929f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:37.326183 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf9162c-ac2f-43eb-9a15-9e99678a929f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:34:37.334546 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-3cf9162c-ac2f-43eb-9a15-9e99678a929f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:34:37.334546 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:34:37.334546 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:34:37.334546 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:34:37.334546 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:34:37.334546 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:34:37.334546 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:34:37.334546 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:34:37.334546 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:34:37.334546 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:34:37.334546 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:34:37.334546 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:34:37.334546 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:34:37.334546 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:34:37.334546 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:34:37.334546 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:34:37.335448 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:34:37.335448 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:34:37.335448 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:34:37.335448 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:34:37.335448 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:34:37.335448 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:34:37.335448 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:34:37.335448 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:34:37.335448 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:34:37.335448 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:34:37.335448 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:34:37.335448 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:34:37.335448 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:34:37.335448 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:34:37.335448 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:34:37.335448 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:34:37.336223 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:34:37.336223 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:34:37.336223 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:34:37.336223 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:34:37.336223 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:34:37.336223 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:34:37.336223 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:34:37.336223 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:34:37.336223 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:34:37.336223 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:34:37.336223 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:34:37.336223 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:34:37.336223 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 86/170] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 21 22:34:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:34:37.336874 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf9162c-ac2f-43eb-9a15-9e99678a929f None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:37.374162 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:37.437826 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d60f70-6349-4ecf-9f15-9710408807fa None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:37.442599 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf9162c-ac2f-43eb-9a15-9e99678a929f None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:37.478101 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a25f18-23b8-49a2-8dc1-ad171aca373f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:37.479299 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a25f18-23b8-49a2-8dc1-ad171aca373f None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:37.479587 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a25f18-23b8-49a2-8dc1-ad171aca373f None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:37.479866 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a25f18-23b8-49a2-8dc1-ad171aca373f None admin] PATH_INFO: `/v3/users/8e9d5ca05dea4d9c9dad9a3a31239f82` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:37.481083 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a25f18-23b8-49a2-8dc1-ad171aca373f None admin] RBAC: Authorizing `identity:get_user(user_id=8e9d5ca05dea4d9c9dad9a3a31239f82)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:37.482719 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a25f18-23b8-49a2-8dc1-ad171aca373f None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:37.484555 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 87/171] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 22:34:37 2022] GET /identity/v3/users/8e9d5ca05dea4d9c9dad9a3a31239f82 => generated 275 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:37.485135 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a25f18-23b8-49a2-8dc1-ad171aca373f None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:37.511217 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d0303e-022a-4670-9d5b-5eca957ee504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:37.511928 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d0303e-022a-4670-9d5b-5eca957ee504 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:37.512226 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d0303e-022a-4670-9d5b-5eca957ee504 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:37.512507 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d0303e-022a-4670-9d5b-5eca957ee504 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:37.513363 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d0303e-022a-4670-9d5b-5eca957ee504 None admin] RBAC: Adding query filter params (role.id=53d3bdbe9cdf4d35bc73854a33df8e45, scope.project.id=6edc20faecff41bf9b7ecc5ac4bc774e, user.id=8e9d5ca05dea4d9c9dad9a3a31239f82) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:37.513678 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d0303e-022a-4670-9d5b-5eca957ee504 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:37.515166 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d0303e-022a-4670-9d5b-5eca957ee504 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:37.526230 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 85/172] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 22:34:37 2022] GET /identity/v3/role_assignments?user.id=8e9d5ca05dea4d9c9dad9a3a31239f82&scope.project.id=6edc20faecff41bf9b7ecc5ac4bc774e&role.id=53d3bdbe9cdf4d35bc73854a33df8e45 => generated 608 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:37.527007 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d0303e-022a-4670-9d5b-5eca957ee504 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:37.595186 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:37.622735 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:37.629282 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a25f18-23b8-49a2-8dc1-ad171aca373f None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:37.691620 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e79b96-c6ec-444e-abec-daf40532a401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:37.692323 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e79b96-c6ec-444e-abec-daf40532a401 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:37.692606 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e79b96-c6ec-444e-abec-daf40532a401 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:37.692877 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e79b96-c6ec-444e-abec-daf40532a401 None admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:37.693651 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e79b96-c6ec-444e-abec-daf40532a401 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:37.693956 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e79b96-c6ec-444e-abec-daf40532a401 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:37.695392 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e79b96-c6ec-444e-abec-daf40532a401 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:37.697099 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d0303e-022a-4670-9d5b-5eca957ee504 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:37.704792 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 88/173] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:34:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:37.710838 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e79b96-c6ec-444e-abec-daf40532a401 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:37.759636 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905df6a3-ced7-4e39-909a-a6087828b932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:37.760987 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905df6a3-ced7-4e39-909a-a6087828b932 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:37.761179 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905df6a3-ced7-4e39-909a-a6087828b932 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:37.761460 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905df6a3-ced7-4e39-909a-a6087828b932 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:37.846202 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e79b96-c6ec-444e-abec-daf40532a401 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:37.863176 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905df6a3-ced7-4e39-909a-a6087828b932 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:37.864657 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905df6a3-ced7-4e39-909a-a6087828b932 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:37.937824 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:37.956908 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8858bde-5f25-415d-9b92-95a9cd59508d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:37.957801 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8858bde-5f25-415d-9b92-95a9cd59508d None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:37.958129 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8858bde-5f25-415d-9b92-95a9cd59508d None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:37.958413 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8858bde-5f25-415d-9b92-95a9cd59508d None admin] PATH_INFO: `/v3/users/8e9d5ca05dea4d9c9dad9a3a31239f82` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:37.959695 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8858bde-5f25-415d-9b92-95a9cd59508d None admin] RBAC: Authorizing `identity:get_user(user_id=8e9d5ca05dea4d9c9dad9a3a31239f82)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:37.961155 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8858bde-5f25-415d-9b92-95a9cd59508d None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:37.965061 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 89/174] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 22:34:37 2022] GET /identity/v3/users/8e9d5ca05dea4d9c9dad9a3a31239f82 => generated 275 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:34:37.966195 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d9754-0977-4a93-a774-da4382360b1a None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:37.966475 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d9754-0977-4a93-a774-da4382360b1a None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:37.966755 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d9754-0977-4a93-a774-da4382360b1a None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:37.968171 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 90/175] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:37.975692 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d9754-0977-4a93-a774-da4382360b1a None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:37.984399 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:37.991998 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 86/176] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:34:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:34:37.993284 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949c08ca-cd04-40cd-8078-9ccd41998b0a None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:37.993632 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949c08ca-cd04-40cd-8078-9ccd41998b0a None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:37.993920 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949c08ca-cd04-40cd-8078-9ccd41998b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:38.011661 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-949c08ca-cd04-40cd-8078-9ccd41998b0a None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:38.012973 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-949c08ca-cd04-40cd-8078-9ccd41998b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:38.088021 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 87/177] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:34:38.088964 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949c08ca-cd04-40cd-8078-9ccd41998b0a None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:38.143125 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d9754-0977-4a93-a774-da4382360b1a None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:38.182916 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ff1861-4bf7-450f-b6a9-dc11caad7f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:38.183610 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff1861-4bf7-450f-b6a9-dc11caad7f3c None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:38.183856 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff1861-4bf7-450f-b6a9-dc11caad7f3c None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:38.184130 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff1861-4bf7-450f-b6a9-dc11caad7f3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:38.184703 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:38.201049 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949c08ca-cd04-40cd-8078-9ccd41998b0a None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:38.238766 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770bb327-8e61-495e-b669-fdc96f0821c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:38.239672 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770bb327-8e61-495e-b669-fdc96f0821c2 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:38.239957 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770bb327-8e61-495e-b669-fdc96f0821c2 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:38.240284 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770bb327-8e61-495e-b669-fdc96f0821c2 None admin] PATH_INFO: `/v3/projects/c07becf9369c4028b8884bbd96515aaf` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:38.241563 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770bb327-8e61-495e-b669-fdc96f0821c2 None admin] RBAC: Authorizing `identity:get_project(project_id=c07becf9369c4028b8884bbd96515aaf)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:38.243123 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770bb327-8e61-495e-b669-fdc96f0821c2 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:38.244867 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 88/178] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:34:38 2022] GET /identity/v3/projects/c07becf9369c4028b8884bbd96515aaf => generated 312 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:38.246091 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351d2c0-bb45-4a54-90e0-ae59f95aa002 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:38.246387 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351d2c0-bb45-4a54-90e0-ae59f95aa002 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:38.246674 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351d2c0-bb45-4a54-90e0-ae59f95aa002 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:38.247925 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 89/179] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:38.252014 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7351d2c0-bb45-4a54-90e0-ae59f95aa002 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:38.261580 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ff1861-4bf7-450f-b6a9-dc11caad7f3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:38.263063 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ff1861-4bf7-450f-b6a9-dc11caad7f3c None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:38.265753 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:38.360057 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7351d2c0-bb45-4a54-90e0-ae59f95aa002 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:38.365021 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 91/180] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:34:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:34:38.366349 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf480c8-b475-4e33-adb0-0323ef091542 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:38.366640 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf480c8-b475-4e33-adb0-0323ef091542 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:38.366949 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf480c8-b475-4e33-adb0-0323ef091542 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:38.368352 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 92/181] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:38.368918 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf480c8-b475-4e33-adb0-0323ef091542 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:38.403796 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4ef073-dda9-45a9-8d27-8f6bd018436a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:38.404589 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ef073-dda9-45a9-8d27-8f6bd018436a None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:38.404878 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ef073-dda9-45a9-8d27-8f6bd018436a None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:38.405154 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ef073-dda9-45a9-8d27-8f6bd018436a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:38.415658 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4ef073-dda9-45a9-8d27-8f6bd018436a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:34:38.423256 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-0d4ef073-dda9-45a9-8d27-8f6bd018436a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:34:38.423256 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:34:38.423256 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:34:38.423256 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:34:38.423256 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:34:38.423256 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:34:38.423256 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:34:38.423256 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:34:38.423256 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:34:38.423256 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:34:38.423256 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:34:38.423256 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:34:38.423256 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:34:38.423256 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:34:38.423256 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:34:38.423256 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:34:38.424173 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:34:38.424173 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:34:38.424173 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:34:38.424173 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:34:38.424173 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:34:38.424173 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:34:38.424173 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:34:38.424173 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:34:38.424173 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:34:38.424173 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:34:38.424173 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:34:38.424173 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:34:38.424173 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:34:38.424173 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:34:38.424173 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:34:38.424173 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:34:38.429983 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:34:38.429983 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:34:38.429983 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:34:38.429983 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:34:38.429983 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:34:38.429983 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:34:38.429983 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:34:38.429983 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:34:38.429983 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:34:38.429983 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:34:38.429983 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:34:38.429983 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:34:38.429983 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 90/182] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 22:34:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:34:38.429983 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdaaf48-5ed9-47b1-8147-cc5a738fd22a None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:38.430678 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdaaf48-5ed9-47b1-8147-cc5a738fd22a None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:38.430678 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdaaf48-5ed9-47b1-8147-cc5a738fd22a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:38.442855 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-6fdaaf48-5ed9-47b1-8147-cc5a738fd22a None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:38.444074 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-6fdaaf48-5ed9-47b1-8147-cc5a738fd22a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:38.453891 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:38.470548 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf480c8-b475-4e33-adb0-0323ef091542 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:38.473577 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:38.506597 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7945d9-3da7-46d0-bed5-ed8807dedc67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:38.507280 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7945d9-3da7-46d0-bed5-ed8807dedc67 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:38.507581 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7945d9-3da7-46d0-bed5-ed8807dedc67 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:38.507850 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7945d9-3da7-46d0-bed5-ed8807dedc67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:38.508676 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7945d9-3da7-46d0-bed5-ed8807dedc67 None admin] RBAC: Adding query filter params (role.id=53d3bdbe9cdf4d35bc73854a33df8e45, scope.project.id=c07becf9369c4028b8884bbd96515aaf, user.id=8e9d5ca05dea4d9c9dad9a3a31239f82) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:38.508986 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7945d9-3da7-46d0-bed5-ed8807dedc67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:38.511049 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7945d9-3da7-46d0-bed5-ed8807dedc67 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:38.512514 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 91/183] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:34:38.516094 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0dc0b-2965-44dc-8fdd-0f1b0569eaec None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:38.516377 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0dc0b-2965-44dc-8fdd-0f1b0569eaec None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:38.516645 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0dc0b-2965-44dc-8fdd-0f1b0569eaec None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:38.526431 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 93/184] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 22:34:38 2022] GET /identity/v3/role_assignments?user.id=8e9d5ca05dea4d9c9dad9a3a31239f82&scope.project.id=c07becf9369c4028b8884bbd96515aaf&role.id=53d3bdbe9cdf4d35bc73854a33df8e45 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:38.526629 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 92/185] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:38.526706 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7945d9-3da7-46d0-bed5-ed8807dedc67 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:38.526847 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e708bf-2d69-485e-a57e-f41fc6f5d5c8 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:38.526847 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e708bf-2d69-485e-a57e-f41fc6f5d5c8 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:38.526847 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e708bf-2d69-485e-a57e-f41fc6f5d5c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:38.550898 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-46e708bf-2d69-485e-a57e-f41fc6f5d5c8 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:38.552150 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-46e708bf-2d69-485e-a57e-f41fc6f5d5c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:38.627985 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 93/186] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:34:38.628860 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e708bf-2d69-485e-a57e-f41fc6f5d5c8 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:38.662212 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7945d9-3da7-46d0-bed5-ed8807dedc67 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:38.707166 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e517bc79-1ff4-44d7-bdd9-439c619b2e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:38.707961 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e517bc79-1ff4-44d7-bdd9-439c619b2e03 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:38.708659 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e517bc79-1ff4-44d7-bdd9-439c619b2e03 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:38.708659 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e517bc79-1ff4-44d7-bdd9-439c619b2e03 None admin] PATH_INFO: `/v3/projects/6edc20faecff41bf9b7ecc5ac4bc774e` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:38.709697 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e517bc79-1ff4-44d7-bdd9-439c619b2e03 None admin] RBAC: Authorizing `identity:get_project(project_id=6edc20faecff41bf9b7ecc5ac4bc774e)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:38.711174 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e517bc79-1ff4-44d7-bdd9-439c619b2e03 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:38.712901 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 94/187] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:34:38 2022] GET /identity/v3/projects/6edc20faecff41bf9b7ecc5ac4bc774e => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:38.714662 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d9fae-69d0-40fa-b6a6-036cd680d819 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:38.715922 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d9fae-69d0-40fa-b6a6-036cd680d819 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:38.716196 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d9fae-69d0-40fa-b6a6-036cd680d819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:38.720762 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:38.734446 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-f32d9fae-69d0-40fa-b6a6-036cd680d819 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:38.736285 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-f32d9fae-69d0-40fa-b6a6-036cd680d819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:38.755034 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e708bf-2d69-485e-a57e-f41fc6f5d5c8 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:38.766442 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:38.789448 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c03cec6-4d23-4944-bbc2-332705314a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:38.790169 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c03cec6-4d23-4944-bbc2-332705314a6b None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:38.790450 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c03cec6-4d23-4944-bbc2-332705314a6b None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:38.790726 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c03cec6-4d23-4944-bbc2-332705314a6b None admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:38.791528 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c03cec6-4d23-4944-bbc2-332705314a6b None admin] RBAC: Adding query filter params (name=admin) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:38.791828 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c03cec6-4d23-4944-bbc2-332705314a6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:38.793204 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c03cec6-4d23-4944-bbc2-332705314a6b None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:38.801364 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 94/188] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:34:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:38.802133 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c03cec6-4d23-4944-bbc2-332705314a6b None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:38.808269 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 95/189] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:34:38.809083 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32d9fae-69d0-40fa-b6a6-036cd680d819 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:38.901893 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c03cec6-4d23-4944-bbc2-332705314a6b None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:38.917033 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32d9fae-69d0-40fa-b6a6-036cd680d819 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:38.934624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8930ec2-6ccf-40a7-a4bd-b76eff42070f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:38.935453 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8930ec2-6ccf-40a7-a4bd-b76eff42070f None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:38.935735 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8930ec2-6ccf-40a7-a4bd-b76eff42070f None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:38.936006 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8930ec2-6ccf-40a7-a4bd-b76eff42070f None admin] PATH_INFO: `/v3/domains/default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:38.937179 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8930ec2-6ccf-40a7-a4bd-b76eff42070f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:38.938704 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8930ec2-6ccf-40a7-a4bd-b76eff42070f None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:38.940406 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 95/190] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:34:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:38.941165 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8930ec2-6ccf-40a7-a4bd-b76eff42070f None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:38.944454 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:38.952896 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaf0c71-c2e4-4439-afb2-a7c596be147b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:38.955200 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf0c71-c2e4-4439-afb2-a7c596be147b None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:38.955200 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf0c71-c2e4-4439-afb2-a7c596be147b None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:38.955200 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf0c71-c2e4-4439-afb2-a7c596be147b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:38.964617 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaf0c71-c2e4-4439-afb2-a7c596be147b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:34:38.972643 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-1eaf0c71-c2e4-4439-afb2-a7c596be147b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:34:38.972643 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:34:38.972643 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:34:38.972643 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:34:38.972643 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:34:38.972643 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:34:38.972643 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:34:38.972643 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:34:38.972643 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:34:38.972643 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:34:38.972643 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:34:38.972643 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:34:38.972643 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:34:38.972643 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:34:38.972643 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:34:38.972643 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:34:38.973514 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:34:38.973514 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:34:38.973514 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:34:38.973514 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:34:38.973514 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:34:38.973514 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:34:38.973514 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:34:38.973514 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:34:38.973514 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:34:38.973514 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:34:38.973514 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:34:38.973514 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:34:38.973514 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:34:38.973514 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:34:38.973514 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:34:38.973514 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:34:38.979964 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:34:38.979964 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:34:38.979964 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:34:38.979964 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:34:38.979964 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:34:38.979964 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:34:38.979964 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:34:38.979964 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:34:38.979964 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:34:38.979964 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:34:38.979964 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:34:38.979964 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:34:38.979964 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 96/191] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 21 22:34:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:34:38.979964 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaf0c71-c2e4-4439-afb2-a7c596be147b None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:39.004870 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:39.039226 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8930ec2-6ccf-40a7-a4bd-b76eff42070f None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:39.072453 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db2e1eb-98fa-40fe-a80a-41263a63285e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:39.073656 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db2e1eb-98fa-40fe-a80a-41263a63285e None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:39.073946 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db2e1eb-98fa-40fe-a80a-41263a63285e None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:39.074225 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db2e1eb-98fa-40fe-a80a-41263a63285e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:39.075090 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db2e1eb-98fa-40fe-a80a-41263a63285e None admin] RBAC: Adding query filter params (role.id=b1849c34fc37410098b94ff687ab0055, scope.project.id=6edc20faecff41bf9b7ecc5ac4bc774e, user.id=8e9d5ca05dea4d9c9dad9a3a31239f82) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:39.075405 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db2e1eb-98fa-40fe-a80a-41263a63285e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:39.076323 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaf0c71-c2e4-4439-afb2-a7c596be147b None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:39.076834 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db2e1eb-98fa-40fe-a80a-41263a63285e None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:39.082727 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 96/192] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 22:34:38 2022] GET /identity/v3/role_assignments?user.id=8e9d5ca05dea4d9c9dad9a3a31239f82&scope.project.id=6edc20faecff41bf9b7ecc5ac4bc774e&role.id=b1849c34fc37410098b94ff687ab0055 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:39.083533 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db2e1eb-98fa-40fe-a80a-41263a63285e None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:39.115379 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a0359-02b4-4ba7-9306-34245b12c469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:39.116274 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a0359-02b4-4ba7-9306-34245b12c469 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:39.116557 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a0359-02b4-4ba7-9306-34245b12c469 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:39.116834 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a0359-02b4-4ba7-9306-34245b12c469 None admin] PATH_INFO: `/v3/users/96a87ea8169f4303b153ccfdfdb6a668` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:39.134284 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977a0359-02b4-4ba7-9306-34245b12c469 None admin] RBAC: Authorizing `identity:get_user(user_id=96a87ea8169f4303b153ccfdfdb6a668)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:39.136432 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977a0359-02b4-4ba7-9306-34245b12c469 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:39.138417 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 97/193] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 22:34:38 2022] GET /identity/v3/users/96a87ea8169f4303b153ccfdfdb6a668 => generated 283 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:39.139072 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a0359-02b4-4ba7-9306-34245b12c469 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:39.174205 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:39.212634 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db2e1eb-98fa-40fe-a80a-41263a63285e None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:39.238892 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:39.253845 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1231f410-0745-409f-83e9-3c3a25b3b2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:39.254640 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1231f410-0745-409f-83e9-3c3a25b3b2ef None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:39.254971 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1231f410-0745-409f-83e9-3c3a25b3b2ef None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:39.255241 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1231f410-0745-409f-83e9-3c3a25b3b2ef None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:39.262311 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a0359-02b4-4ba7-9306-34245b12c469 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:39.267790 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1231f410-0745-409f-83e9-3c3a25b3b2ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:34:39.276783 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-1231f410-0745-409f-83e9-3c3a25b3b2ef None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:34:39.276783 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:34:39.276783 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:34:39.276783 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:34:39.276783 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:34:39.276783 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:34:39.276783 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:34:39.276783 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:34:39.276783 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:34:39.276783 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:34:39.276783 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:34:39.276783 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:34:39.276783 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:34:39.276783 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:34:39.276783 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:34:39.276783 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:34:39.279034 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:34:39.279034 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:34:39.279034 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:34:39.279034 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:34:39.279034 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:34:39.279034 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:34:39.279034 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:34:39.279034 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:34:39.279034 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:34:39.279034 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:34:39.279034 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:34:39.279034 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:34:39.279034 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:34:39.279034 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:34:39.279034 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:34:39.279034 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:34:39.279826 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:34:39.279826 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:34:39.279826 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:34:39.279826 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:34:39.279826 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:34:39.279826 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:34:39.279826 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:34:39.279826 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:34:39.279826 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:34:39.279826 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:34:39.279826 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:34:39.279826 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:34:39.279826 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 97/194] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:34:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:34:39.279826 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1231f410-0745-409f-83e9-3c3a25b3b2ef None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:39.301048 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e32037-34a1-4cfc-92c3-f7333f83d2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:39.301734 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e32037-34a1-4cfc-92c3-f7333f83d2bd None admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:39.302103 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e32037-34a1-4cfc-92c3-f7333f83d2bd None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:39.302293 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e32037-34a1-4cfc-92c3-f7333f83d2bd None admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:39.303042 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e32037-34a1-4cfc-92c3-f7333f83d2bd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:39.305197 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e32037-34a1-4cfc-92c3-f7333f83d2bd None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:39.387844 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1231f410-0745-409f-83e9-3c3a25b3b2ef None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:39.406877 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 98/195] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 21 22:34:39 2022] POST /identity/v3/users => generated 297 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:39.407791 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e32037-34a1-4cfc-92c3-f7333f83d2bd None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:39.419523 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:39.422211 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe8fe74-2b66-4854-9b1b-efad8794554d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:39.422842 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe8fe74-2b66-4854-9b1b-efad8794554d None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:39.423129 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe8fe74-2b66-4854-9b1b-efad8794554d None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:39.423405 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe8fe74-2b66-4854-9b1b-efad8794554d None admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:39.424177 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe8fe74-2b66-4854-9b1b-efad8794554d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:39.424488 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe8fe74-2b66-4854-9b1b-efad8794554d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:39.425918 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe8fe74-2b66-4854-9b1b-efad8794554d None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:39.436168 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 98/196] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:34:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:39.436943 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe8fe74-2b66-4854-9b1b-efad8794554d None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:39.542493 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e32037-34a1-4cfc-92c3-f7333f83d2bd None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:39.563335 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe8fe74-2b66-4854-9b1b-efad8794554d None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:39.583662 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3995279-6e5a-481e-b09f-3e2ec09ac2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:39.584334 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3995279-6e5a-481e-b09f-3e2ec09ac2b2 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:39.584616 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3995279-6e5a-481e-b09f-3e2ec09ac2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:39.584892 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3995279-6e5a-481e-b09f-3e2ec09ac2b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:39.599262 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:39.601434 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29403e71-af62-42fa-97d4-4a29680dcdfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:39.602111 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29403e71-af62-42fa-97d4-4a29680dcdfe None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:39.602388 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29403e71-af62-42fa-97d4-4a29680dcdfe None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:39.602664 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29403e71-af62-42fa-97d4-4a29680dcdfe None admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:39.603455 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29403e71-af62-42fa-97d4-4a29680dcdfe None admin] RBAC: Adding query filter params (name=reader) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:39.604549 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29403e71-af62-42fa-97d4-4a29680dcdfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:39.605941 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29403e71-af62-42fa-97d4-4a29680dcdfe None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:39.614571 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 99/197] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 22:34:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:39.615418 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29403e71-af62-42fa-97d4-4a29680dcdfe None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:39.670635 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3995279-6e5a-481e-b09f-3e2ec09ac2b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:39.672489 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3995279-6e5a-481e-b09f-3e2ec09ac2b2 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:39.681381 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:39.742870 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29403e71-af62-42fa-97d4-4a29680dcdfe None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:39.792385 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0df234-d39e-4304-b0b8-f75b2f163e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:39.793813 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0df234-d39e-4304-b0b8-f75b2f163e0e None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:39.794666 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0df234-d39e-4304-b0b8-f75b2f163e0e None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:39.795252 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0df234-d39e-4304-b0b8-f75b2f163e0e None admin] PATH_INFO: `/v3/users/96a87ea8169f4303b153ccfdfdb6a668` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:39.796981 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0df234-d39e-4304-b0b8-f75b2f163e0e None admin] RBAC: Authorizing `identity:get_user(user_id=96a87ea8169f4303b153ccfdfdb6a668)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:39.799307 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0df234-d39e-4304-b0b8-f75b2f163e0e None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:39.801172 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 100/198] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 22:34:39 2022] GET /identity/v3/users/96a87ea8169f4303b153ccfdfdb6a668 => generated 283 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:39.802025 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 99/199] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:34:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:34:39.802252 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0df234-d39e-4304-b0b8-f75b2f163e0e None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:39.812070 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3995279-6e5a-481e-b09f-3e2ec09ac2b2 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:39.866057 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:39.916674 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0df234-d39e-4304-b0b8-f75b2f163e0e None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:39.938802 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:39.941011 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3995279-6e5a-481e-b09f-3e2ec09ac2b2 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:39.962626 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca6337a-3f54-451c-99a8-e4a18d558fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:39.963752 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca6337a-3f54-451c-99a8-e4a18d558fbe None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:39.964039 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca6337a-3f54-451c-99a8-e4a18d558fbe None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:39.964317 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca6337a-3f54-451c-99a8-e4a18d558fbe None admin] PATH_INFO: `/v3/users/a6860360a136470c9d8dcc9ac43849f9` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:39.965786 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca6337a-3f54-451c-99a8-e4a18d558fbe None admin] RBAC: Authorizing `identity:get_user(user_id=a6860360a136470c9d8dcc9ac43849f9)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:39.967178 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca6337a-3f54-451c-99a8-e4a18d558fbe None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:39.970430 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 101/200] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 22:34:39 2022] GET /identity/v3/users/a6860360a136470c9d8dcc9ac43849f9 => generated 289 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:39.971408 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca6337a-3f54-451c-99a8-e4a18d558fbe None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:39.992937 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf09ee3-8502-4d69-b7a6-ac8ad1c44337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:39.994973 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf09ee3-8502-4d69-b7a6-ac8ad1c44337 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:39.995557 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf09ee3-8502-4d69-b7a6-ac8ad1c44337 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:39.995981 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf09ee3-8502-4d69-b7a6-ac8ad1c44337 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:40.090741 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca6337a-3f54-451c-99a8-e4a18d558fbe None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:40.091817 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf09ee3-8502-4d69-b7a6-ac8ad1c44337 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:40.094406 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf09ee3-8502-4d69-b7a6-ac8ad1c44337 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:40.127567 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1721a62f-46e9-4d08-87ea-2b8354244dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:40.128582 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1721a62f-46e9-4d08-87ea-2b8354244dd4 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:40.129009 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1721a62f-46e9-4d08-87ea-2b8354244dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:40.129478 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1721a62f-46e9-4d08-87ea-2b8354244dd4 None admin] PATH_INFO: `/v3/projects/a54cbea0067945899d2c8c75b7973660` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:40.130918 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1721a62f-46e9-4d08-87ea-2b8354244dd4 None admin] RBAC: Authorizing `identity:get_project(project_id=a54cbea0067945899d2c8c75b7973660)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:40.132516 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1721a62f-46e9-4d08-87ea-2b8354244dd4 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:40.140132 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 102/201] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:34:39 2022] GET /identity/v3/projects/a54cbea0067945899d2c8c75b7973660 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:40.141090 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1721a62f-46e9-4d08-87ea-2b8354244dd4 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:40.146581 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:40.229898 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 100/202] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:34:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 22:34:40.229898 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf09ee3-8502-4d69-b7a6-ac8ad1c44337 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:40.230163 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:40.256017 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1721a62f-46e9-4d08-87ea-2b8354244dd4 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:40.296563 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff31585-4c94-47a1-ab4d-e1f4dcdb57a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:40.297649 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff31585-4c94-47a1-ab4d-e1f4dcdb57a5 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:40.298195 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff31585-4c94-47a1-ab4d-e1f4dcdb57a5 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:40.298673 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff31585-4c94-47a1-ab4d-e1f4dcdb57a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:40.361003 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf09ee3-8502-4d69-b7a6-ac8ad1c44337 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:40.377046 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff31585-4c94-47a1-ab4d-e1f4dcdb57a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:40.379098 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff31585-4c94-47a1-ab4d-e1f4dcdb57a5 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:40.407802 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9540f688-c959-410d-af0d-32c414abe51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:40.408737 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540f688-c959-410d-af0d-32c414abe51c None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:40.409026 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540f688-c959-410d-af0d-32c414abe51c None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:40.409307 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540f688-c959-410d-af0d-32c414abe51c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:40.410416 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9540f688-c959-410d-af0d-32c414abe51c None admin] RBAC: Adding query filter params (role.id=d1ddee590a964ff1bbb4ee8716173021, scope.project.id=a54cbea0067945899d2c8c75b7973660, user.id=96a87ea8169f4303b153ccfdfdb6a668) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:40.410737 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9540f688-c959-410d-af0d-32c414abe51c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:40.412901 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9540f688-c959-410d-af0d-32c414abe51c None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:40.420529 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 101/203] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 22:34:40 2022] GET /identity/v3/role_assignments?user.id=96a87ea8169f4303b153ccfdfdb6a668&scope.project.id=a54cbea0067945899d2c8c75b7973660&role.id=d1ddee590a964ff1bbb4ee8716173021 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:40.421314 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9540f688-c959-410d-af0d-32c414abe51c None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:40.425198 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:40.444651 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:40.535527 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 103/204] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:34:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:34:40.542333 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff31585-4c94-47a1-ab4d-e1f4dcdb57a5 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:40.660284 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9540f688-c959-410d-af0d-32c414abe51c None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:40.670393 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff31585-4c94-47a1-ab4d-e1f4dcdb57a5 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:40.709175 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b4cbdc-8ca0-4192-a875-8024c40c2be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:40.710038 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b4cbdc-8ca0-4192-a875-8024c40c2be4 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:40.710323 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b4cbdc-8ca0-4192-a875-8024c40c2be4 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:40.710601 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b4cbdc-8ca0-4192-a875-8024c40c2be4 None admin] PATH_INFO: `/v3/projects/6edc20faecff41bf9b7ecc5ac4bc774e` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:40.711863 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b4cbdc-8ca0-4192-a875-8024c40c2be4 None admin] RBAC: Authorizing `identity:get_project(project_id=6edc20faecff41bf9b7ecc5ac4bc774e)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:40.713852 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b4cbdc-8ca0-4192-a875-8024c40c2be4 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:40.715921 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 104/205] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:34:40 2022] GET /identity/v3/projects/6edc20faecff41bf9b7ecc5ac4bc774e => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:40.721460 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b4cbdc-8ca0-4192-a875-8024c40c2be4 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:40.725257 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1135e1-db56-4a11-b630-c4c417ce436d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:40.726134 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1135e1-db56-4a11-b630-c4c417ce436d None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:40.726419 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1135e1-db56-4a11-b630-c4c417ce436d None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:40.726700 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1135e1-db56-4a11-b630-c4c417ce436d None admin] PATH_INFO: `/v3/projects/a54cbea0067945899d2c8c75b7973660` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:40.731524 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1135e1-db56-4a11-b630-c4c417ce436d None admin] RBAC: Authorizing `identity:get_project(project_id=a54cbea0067945899d2c8c75b7973660)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:40.732993 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1135e1-db56-4a11-b630-c4c417ce436d None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:40.734955 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 102/206] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:34:40 2022] GET /identity/v3/projects/a54cbea0067945899d2c8c75b7973660 => generated 302 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:40.741510 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1135e1-db56-4a11-b630-c4c417ce436d None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:40.779138 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:40.886986 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b4cbdc-8ca0-4192-a875-8024c40c2be4 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:40.924622 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509037f-e22d-4856-be3b-5dba66df8fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:40.925226 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509037f-e22d-4856-be3b-5dba66df8fbe None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:40.925522 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509037f-e22d-4856-be3b-5dba66df8fbe None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:40.925811 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509037f-e22d-4856-be3b-5dba66df8fbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:40.926653 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f509037f-e22d-4856-be3b-5dba66df8fbe None admin] RBAC: Adding query filter params (role.id=6ea16f1a323a473e81c1b161134d7512, scope.project.id=6edc20faecff41bf9b7ecc5ac4bc774e, user.id=a6860360a136470c9d8dcc9ac43849f9) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:40.927138 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f509037f-e22d-4856-be3b-5dba66df8fbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:40.928598 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f509037f-e22d-4856-be3b-5dba66df8fbe None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:40.935266 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 105/207] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 22:34:40 2022] GET /identity/v3/role_assignments?user.id=a6860360a136470c9d8dcc9ac43849f9&scope.project.id=6edc20faecff41bf9b7ecc5ac4bc774e&role.id=6ea16f1a323a473e81c1b161134d7512 => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:40.952199 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:40.999331 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1135e1-db56-4a11-b630-c4c417ce436d None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:41.038483 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074294de-6ac5-420b-898f-0da320aeb1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:41.039206 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074294de-6ac5-420b-898f-0da320aeb1df None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:41.039589 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074294de-6ac5-420b-898f-0da320aeb1df None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:41.039868 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074294de-6ac5-420b-898f-0da320aeb1df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:41.040739 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074294de-6ac5-420b-898f-0da320aeb1df None admin] RBAC: Adding query filter params (role.id=b1849c34fc37410098b94ff687ab0055, scope.project.id=a54cbea0067945899d2c8c75b7973660, user.id=96a87ea8169f4303b153ccfdfdb6a668) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:41.041056 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074294de-6ac5-420b-898f-0da320aeb1df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:41.042549 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074294de-6ac5-420b-898f-0da320aeb1df None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:41.049115 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 103/208] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 22:34:40 2022] GET /identity/v3/role_assignments?user.id=96a87ea8169f4303b153ccfdfdb6a668&scope.project.id=a54cbea0067945899d2c8c75b7973660&role.id=b1849c34fc37410098b94ff687ab0055 => generated 259 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:41.718665 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de608e64-690c-409e-8e47-b7c5bf461e3e None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:41.719193 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de608e64-690c-409e-8e47-b7c5bf461e3e None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:41.719269 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de608e64-690c-409e-8e47-b7c5bf461e3e None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:41.720932 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 106/209] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:41.723535 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd528970-59b4-4ec7-9ca3-322054b37eae None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:41.723947 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd528970-59b4-4ec7-9ca3-322054b37eae None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:41.724329 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd528970-59b4-4ec7-9ca3-322054b37eae None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:41.726351 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d5e1af-3a15-4712-92e4-3e369214afb1 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:41.726489 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 104/210] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:41.726639 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d5e1af-3a15-4712-92e4-3e369214afb1 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:41.726953 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d5e1af-3a15-4712-92e4-3e369214afb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:41.731853 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd72c202-8db3-42a4-856d-275cd6f1dd73 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:41.732406 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd72c202-8db3-42a4-856d-275cd6f1dd73 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:41.732882 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd72c202-8db3-42a4-856d-275cd6f1dd73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:41.744856 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-b4d5e1af-3a15-4712-92e4-3e369214afb1 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:41.746095 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-b4d5e1af-3a15-4712-92e4-3e369214afb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:41.753355 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-fd72c202-8db3-42a4-856d-275cd6f1dd73 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:41.755198 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-fd72c202-8db3-42a4-856d-275cd6f1dd73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:41.817650 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 107/211] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:34:41.840673 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 105/212] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:34:41.841371 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d5e1af-3a15-4712-92e4-3e369214afb1 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:41.864382 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:41.868288 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd72c202-8db3-42a4-856d-275cd6f1dd73 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:41.936256 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:41.971175 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd72c202-8db3-42a4-856d-275cd6f1dd73 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:41.975890 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d5e1af-3a15-4712-92e4-3e369214afb1 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:42.007437 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dc291b-d5a6-491b-9d72-70a6887dd7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:42.008241 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc291b-d5a6-491b-9d72-70a6887dd7db None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:42.008589 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc291b-d5a6-491b-9d72-70a6887dd7db None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:42.008946 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc291b-d5a6-491b-9d72-70a6887dd7db None admin] PATH_INFO: `/v3/domains/default` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:42.010330 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dc291b-d5a6-491b-9d72-70a6887dd7db None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:42.011864 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dc291b-d5a6-491b-9d72-70a6887dd7db None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:42.013539 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 106/213] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:34:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:34:42.019100 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dc291b-d5a6-491b-9d72-70a6887dd7db None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:42.021280 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8d8a92-e3d9-400d-a97d-2789a71e42f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:42.023438 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d8a92-e3d9-400d-a97d-2789a71e42f5 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:42.023914 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d8a92-e3d9-400d-a97d-2789a71e42f5 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:42.024373 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d8a92-e3d9-400d-a97d-2789a71e42f5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:42.038848 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8d8a92-e3d9-400d-a97d-2789a71e42f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:34:42.050173 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-3d8d8a92-e3d9-400d-a97d-2789a71e42f5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:34:42.050173 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:34:42.050173 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:34:42.050173 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:34:42.050173 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:34:42.050173 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:34:42.050173 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:34:42.050173 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:34:42.050173 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:34:42.050173 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:34:42.050173 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:34:42.050173 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:34:42.050173 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:34:42.050173 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:34:42.050173 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:34:42.050173 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:34:42.051080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:34:42.051080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:34:42.051080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:34:42.051080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:34:42.051080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:34:42.051080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:34:42.051080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:34:42.051080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:34:42.051080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:34:42.051080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:34:42.051080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:34:42.051080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:34:42.051080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:34:42.051080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:34:42.051080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:34:42.051080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:34:42.051881 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:34:42.051881 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:34:42.051881 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:34:42.051881 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:34:42.051881 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:34:42.051881 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:34:42.051881 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:34:42.051881 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:34:42.051881 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:34:42.051881 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:34:42.051881 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:34:42.051881 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:34:42.054584 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 108/214] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:34:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:34:42.060142 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8d8a92-e3d9-400d-a97d-2789a71e42f5 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:42.145751 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dc291b-d5a6-491b-9d72-70a6887dd7db None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:42.152248 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:42.183022 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8d8a92-e3d9-400d-a97d-2789a71e42f5 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:42.186737 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5a40bd-4f85-479d-a6ec-24635543733b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:42.186917 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5a40bd-4f85-479d-a6ec-24635543733b None admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:42.187249 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5a40bd-4f85-479d-a6ec-24635543733b None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:42.187610 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5a40bd-4f85-479d-a6ec-24635543733b None admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:42.188458 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5a40bd-4f85-479d-a6ec-24635543733b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:42.189963 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5a40bd-4f85-479d-a6ec-24635543733b None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:42.196744 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:42.224044 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a5d04c-0931-4df3-908a-a711bd8fffc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:42.224731 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a5d04c-0931-4df3-908a-a711bd8fffc8 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:42.225027 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a5d04c-0931-4df3-908a-a711bd8fffc8 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:42.225319 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a5d04c-0931-4df3-908a-a711bd8fffc8 None admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:42.226063 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a5d04c-0931-4df3-908a-a711bd8fffc8 None admin] RBAC: Adding query filter params (name=member) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:42.226375 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a5d04c-0931-4df3-908a-a711bd8fffc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:42.227849 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a5d04c-0931-4df3-908a-a711bd8fffc8 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:42.236957 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 109/215] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 22:34:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:34:42.242306 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a5d04c-0931-4df3-908a-a711bd8fffc8 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:42.282002 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 107/216] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 21 22:34:42 2022] POST /identity/v3/users => generated 297 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:34:42.373350 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a5d04c-0931-4df3-908a-a711bd8fffc8 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:42.413418 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2413a30-3063-4458-be31-04bd54b2dbe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:42.414670 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2413a30-3063-4458-be31-04bd54b2dbe3 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:42.415202 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2413a30-3063-4458-be31-04bd54b2dbe3 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:42.415679 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2413a30-3063-4458-be31-04bd54b2dbe3 None admin] PATH_INFO: `/v3/users/ded11401a1b9424ba8a649bb8fc842f9` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:42.428984 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:42.434278 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2413a30-3063-4458-be31-04bd54b2dbe3 None admin] RBAC: Authorizing `identity:get_user(user_id=ded11401a1b9424ba8a649bb8fc842f9)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:42.436129 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2413a30-3063-4458-be31-04bd54b2dbe3 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:42.438160 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 110/217] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 22:34:42 2022] GET /identity/v3/users/ded11401a1b9424ba8a649bb8fc842f9 => generated 297 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:42.443825 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5a40bd-4f85-479d-a6ec-24635543733b None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:42.554124 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5a40bd-4f85-479d-a6ec-24635543733b None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:42.593306 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642911a5-8706-42b5-9c6f-ee45c62449c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:42.594323 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642911a5-8706-42b5-9c6f-ee45c62449c9 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:42.594835 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642911a5-8706-42b5-9c6f-ee45c62449c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:42.595315 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642911a5-8706-42b5-9c6f-ee45c62449c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:42.658162 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:42.669391 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642911a5-8706-42b5-9c6f-ee45c62449c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:42.670946 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642911a5-8706-42b5-9c6f-ee45c62449c9 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:42.700274 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d1c2d1-7bc2-437e-b1ae-04f94cac0715 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:42.700567 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d1c2d1-7bc2-437e-b1ae-04f94cac0715 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:42.700898 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d1c2d1-7bc2-437e-b1ae-04f94cac0715 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:42.702287 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 111/218] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:34:42.708130 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7baf5b-9496-4040-a5a3-d3d23e300984 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:42.708424 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7baf5b-9496-4040-a5a3-d3d23e300984 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:42.708707 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7baf5b-9496-4040-a5a3-d3d23e300984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:42.732210 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-2b7baf5b-9496-4040-a5a3-d3d23e300984 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:42.733850 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-2b7baf5b-9496-4040-a5a3-d3d23e300984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:42.798825 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 108/219] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:34:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:34:42.806589 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642911a5-8706-42b5-9c6f-ee45c62449c9 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:42.842817 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 112/220] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:34:42.877809 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7baf5b-9496-4040-a5a3-d3d23e300984 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:42.979178 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:42.981723 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642911a5-8706-42b5-9c6f-ee45c62449c9 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:43.050630 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f35c43-d85b-4fc5-9759-59ddbfa27b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:43.051524 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f35c43-d85b-4fc5-9759-59ddbfa27b91 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:43.051803 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f35c43-d85b-4fc5-9759-59ddbfa27b91 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:43.052073 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f35c43-d85b-4fc5-9759-59ddbfa27b91 None admin] PATH_INFO: `/v3/projects/a54cbea0067945899d2c8c75b7973660` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:43.053459 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f35c43-d85b-4fc5-9759-59ddbfa27b91 None admin] RBAC: Authorizing `identity:get_project(project_id=a54cbea0067945899d2c8c75b7973660)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:43.054920 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f35c43-d85b-4fc5-9759-59ddbfa27b91 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:43.058597 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 109/221] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:34:42 2022] GET /identity/v3/projects/a54cbea0067945899d2c8c75b7973660 => generated 302 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:43.063424 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f35c43-d85b-4fc5-9759-59ddbfa27b91 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:43.072311 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7baf5b-9496-4040-a5a3-d3d23e300984 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:43.107914 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:43.122831 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9e8981-0d0a-4d9c-b110-86f4648a8f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:43.123620 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e8981-0d0a-4d9c-b110-86f4648a8f03 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:43.123900 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e8981-0d0a-4d9c-b110-86f4648a8f03 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:43.124176 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e8981-0d0a-4d9c-b110-86f4648a8f03 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:43.152660 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9e8981-0d0a-4d9c-b110-86f4648a8f03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:34:43.160264 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-5f9e8981-0d0a-4d9c-b110-86f4648a8f03 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:34:43.160264 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:34:43.160264 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:34:43.160264 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:34:43.160264 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:34:43.160264 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:34:43.160264 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:34:43.160264 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:34:43.160264 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:34:43.160264 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:34:43.160264 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:34:43.160264 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:34:43.160264 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:34:43.160264 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:34:43.160264 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:34:43.160264 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:34:43.161130 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:34:43.161130 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:34:43.161130 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:34:43.161130 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:34:43.161130 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:34:43.161130 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:34:43.161130 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:34:43.161130 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:34:43.161130 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:34:43.161130 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:34:43.161130 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:34:43.161130 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:34:43.161130 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:34:43.161130 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:34:43.161130 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:34:43.161130 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:34:43.161931 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:34:43.161931 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:34:43.161931 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:34:43.161931 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:34:43.161931 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:34:43.161931 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:34:43.161931 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:34:43.161931 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:34:43.161931 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:34:43.161931 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:34:43.161931 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:34:43.161931 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:34:43.161931 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 113/222] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 22:34:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:34:43.169147 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9e8981-0d0a-4d9c-b110-86f4648a8f03 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:43.266731 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f35c43-d85b-4fc5-9759-59ddbfa27b91 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:43.333178 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b47d1dc-621c-48c5-86ab-9e1980fbdd95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:43.333871 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47d1dc-621c-48c5-86ab-9e1980fbdd95 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:43.334159 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47d1dc-621c-48c5-86ab-9e1980fbdd95 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:43.334444 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47d1dc-621c-48c5-86ab-9e1980fbdd95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:43.335319 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b47d1dc-621c-48c5-86ab-9e1980fbdd95 None admin] RBAC: Adding query filter params (role.id=53d3bdbe9cdf4d35bc73854a33df8e45, scope.project.id=a54cbea0067945899d2c8c75b7973660, user.id=ded11401a1b9424ba8a649bb8fc842f9) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:43.335635 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b47d1dc-621c-48c5-86ab-9e1980fbdd95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:43.337618 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b47d1dc-621c-48c5-86ab-9e1980fbdd95 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:43.344659 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 110/223] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 22:34:43 2022] GET /identity/v3/role_assignments?user.id=ded11401a1b9424ba8a649bb8fc842f9&scope.project.id=a54cbea0067945899d2c8c75b7973660&role.id=53d3bdbe9cdf4d35bc73854a33df8e45 => generated 259 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:34:43.346126 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc10253-fed1-4723-88d9-7370d5651280 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:43.346418 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc10253-fed1-4723-88d9-7370d5651280 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:43.346700 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc10253-fed1-4723-88d9-7370d5651280 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:43.348150 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 111/224] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:43.352758 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9e8981-0d0a-4d9c-b110-86f4648a8f03 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:43.354371 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec0e9f-7df1-4026-84e9-11de57cda088 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:43.354758 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec0e9f-7df1-4026-84e9-11de57cda088 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:43.355058 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec0e9f-7df1-4026-84e9-11de57cda088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:43.381767 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-85ec0e9f-7df1-4026-84e9-11de57cda088 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:43.383881 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-85ec0e9f-7df1-4026-84e9-11de57cda088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:43.435427 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7cc56c-7d23-4136-a863-ba1b4fc3dc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:43.436220 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7cc56c-7d23-4136-a863-ba1b4fc3dc61 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:43.436512 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7cc56c-7d23-4136-a863-ba1b4fc3dc61 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:43.436790 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7cc56c-7d23-4136-a863-ba1b4fc3dc61 None admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:43.437559 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7cc56c-7d23-4136-a863-ba1b4fc3dc61 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:43.437870 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7cc56c-7d23-4136-a863-ba1b4fc3dc61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:43.439322 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7cc56c-7d23-4136-a863-ba1b4fc3dc61 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:43.447160 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:43.461387 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 114/225] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:34:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:43.469069 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7cc56c-7d23-4136-a863-ba1b4fc3dc61 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:43.535828 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 112/226] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:34:43.544497 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:43.573632 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec0e9f-7df1-4026-84e9-11de57cda088 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:43.597553 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7cc56c-7d23-4136-a863-ba1b4fc3dc61 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:43.635863 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6399da-85b8-4364-9823-ee4d7d78d902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:43.636893 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6399da-85b8-4364-9823-ee4d7d78d902 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:43.637284 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6399da-85b8-4364-9823-ee4d7d78d902 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:43.637654 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6399da-85b8-4364-9823-ee4d7d78d902 None admin] PATH_INFO: `/v3/users/96a87ea8169f4303b153ccfdfdb6a668` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:43.639014 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6399da-85b8-4364-9823-ee4d7d78d902 None admin] RBAC: Authorizing `identity:get_user(user_id=96a87ea8169f4303b153ccfdfdb6a668)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:43.640548 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6399da-85b8-4364-9823-ee4d7d78d902 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:43.642310 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 115/227] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 22:34:43 2022] GET /identity/v3/users/96a87ea8169f4303b153ccfdfdb6a668 => generated 283 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:43.649680 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6399da-85b8-4364-9823-ee4d7d78d902 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:43.690559 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec0e9f-7df1-4026-84e9-11de57cda088 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:43.727721 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c51bf5-6378-43e1-9227-56e67f104fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:43.729345 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c51bf5-6378-43e1-9227-56e67f104fa9 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:43.729808 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c51bf5-6378-43e1-9227-56e67f104fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:43.730231 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c51bf5-6378-43e1-9227-56e67f104fa9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:43.742331 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c51bf5-6378-43e1-9227-56e67f104fa9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:34:43.751510 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-43c51bf5-6378-43e1-9227-56e67f104fa9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:34:43.751510 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:34:43.751510 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:34:43.751510 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:34:43.751510 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:34:43.751510 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:34:43.751510 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:34:43.751510 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:34:43.751510 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:34:43.751510 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:34:43.751510 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:34:43.751510 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:34:43.751510 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:34:43.751510 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:34:43.751510 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:34:43.751510 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:34:43.752604 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:34:43.752604 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:34:43.752604 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:34:43.752604 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:34:43.752604 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:34:43.752604 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:34:43.752604 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:34:43.752604 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:34:43.752604 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:34:43.752604 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:34:43.752604 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:34:43.752604 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:34:43.752604 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:34:43.752604 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:34:43.752604 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:34:43.752604 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:34:43.753599 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:34:43.753599 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:34:43.753599 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:34:43.753599 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:34:43.753599 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:34:43.753599 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:34:43.753599 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:34:43.753599 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:34:43.753599 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:34:43.753599 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:34:43.753599 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:34:43.753599 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:34:43.761041 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 113/228] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 21 22:34:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:34:43.762402 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c51bf5-6378-43e1-9227-56e67f104fa9 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:43.794756 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:43.827850 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6399da-85b8-4364-9823-ee4d7d78d902 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:43.859015 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:43.877765 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be2d9dd-8528-42bd-84e8-d21b9e6bc6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:43.878641 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be2d9dd-8528-42bd-84e8-d21b9e6bc6a4 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:43.878948 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be2d9dd-8528-42bd-84e8-d21b9e6bc6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:43.879221 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be2d9dd-8528-42bd-84e8-d21b9e6bc6a4 None admin] PATH_INFO: `/v3/projects/a54cbea0067945899d2c8c75b7973660` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:43.887627 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be2d9dd-8528-42bd-84e8-d21b9e6bc6a4 None admin] RBAC: Authorizing `identity:get_project(project_id=a54cbea0067945899d2c8c75b7973660)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:43.889076 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be2d9dd-8528-42bd-84e8-d21b9e6bc6a4 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:43.892472 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 116/229] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:34:43 2022] GET /identity/v3/projects/a54cbea0067945899d2c8c75b7973660 => generated 302 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:43.897372 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be2d9dd-8528-42bd-84e8-d21b9e6bc6a4 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:43.977985 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c51bf5-6378-43e1-9227-56e67f104fa9 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:44.014603 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be2d9dd-8528-42bd-84e8-d21b9e6bc6a4 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:44.052675 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d2c906-8c63-4ca0-8504-b6e4566f02db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:44.054345 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d2c906-8c63-4ca0-8504-b6e4566f02db None admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:44.055154 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d2c906-8c63-4ca0-8504-b6e4566f02db None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:44.056123 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d2c906-8c63-4ca0-8504-b6e4566f02db None admin] PATH_INFO: `/v3/projects/a54cbea0067945899d2c8c75b7973660/users/96a87ea8169f4303b153ccfdfdb6a668/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:44.056964 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa76cdb5-53c2-464a-8204-d0a866e90550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:44.059533 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76cdb5-53c2-464a-8204-d0a866e90550 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:44.060015 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76cdb5-53c2-464a-8204-d0a866e90550 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:44.060908 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d2c906-8c63-4ca0-8504-b6e4566f02db None admin] RBAC: Authorizing `identity:create_grant(project_id=a54cbea0067945899d2c8c75b7973660, user_id=96a87ea8169f4303b153ccfdfdb6a668, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:44.061385 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76cdb5-53c2-464a-8204-d0a866e90550 None admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:44.062332 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa76cdb5-53c2-464a-8204-d0a866e90550 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:44.064094 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d2c906-8c63-4ca0-8504-b6e4566f02db None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:44.064615 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa76cdb5-53c2-464a-8204-d0a866e90550 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:44.066161 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa76cdb5-53c2-464a-8204-d0a866e90550 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:44.079813 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 114/230] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:34:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:34:44.087539 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa76cdb5-53c2-464a-8204-d0a866e90550 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:44.155310 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 117/231] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 21 22:34:43 2022] PUT /identity/v3/projects/a54cbea0067945899d2c8c75b7973660/users/96a87ea8169f4303b153ccfdfdb6a668/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:34:44.199923 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:44.216499 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa76cdb5-53c2-464a-8204-d0a866e90550 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:44.278517 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2664a598-b09d-4699-942f-41503cb348f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:44.279496 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664a598-b09d-4699-942f-41503cb348f0 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:44.279835 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664a598-b09d-4699-942f-41503cb348f0 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:44.280116 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664a598-b09d-4699-942f-41503cb348f0 None admin] PATH_INFO: `/v3/users/96a87ea8169f4303b153ccfdfdb6a668` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:44.281478 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2664a598-b09d-4699-942f-41503cb348f0 None admin] RBAC: Authorizing `identity:get_user(user_id=96a87ea8169f4303b153ccfdfdb6a668)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:44.282975 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2664a598-b09d-4699-942f-41503cb348f0 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:44.284825 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 115/232] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 22:34:44 2022] GET /identity/v3/users/96a87ea8169f4303b153ccfdfdb6a668 => generated 283 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:34:44.290676 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d2c906-8c63-4ca0-8504-b6e4566f02db None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:44.348123 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:44.407027 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d2c906-8c63-4ca0-8504-b6e4566f02db None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:44.444270 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc29529-fe84-42cb-a112-91c6e6705c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:44.445154 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc29529-fe84-42cb-a112-91c6e6705c02 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:44.445441 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc29529-fe84-42cb-a112-91c6e6705c02 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:44.445721 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc29529-fe84-42cb-a112-91c6e6705c02 None admin] PATH_INFO: `/v3/projects/a54cbea0067945899d2c8c75b7973660` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:44.446966 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc29529-fe84-42cb-a112-91c6e6705c02 None admin] RBAC: Authorizing `identity:get_project(project_id=a54cbea0067945899d2c8c75b7973660)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:44.448449 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc29529-fe84-42cb-a112-91c6e6705c02 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:44.454902 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 118/233] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:34:44 2022] GET /identity/v3/projects/a54cbea0067945899d2c8c75b7973660 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:44.456358 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2664a598-b09d-4699-942f-41503cb348f0 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:44.602877 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2664a598-b09d-4699-942f-41503cb348f0 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:44.649646 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5753f0-3ed2-4f5d-a252-c24030cb1e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:44.651140 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5753f0-3ed2-4f5d-a252-c24030cb1e2f None admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:44.651429 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5753f0-3ed2-4f5d-a252-c24030cb1e2f None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:44.651705 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5753f0-3ed2-4f5d-a252-c24030cb1e2f None admin] PATH_INFO: `/v3/projects/a54cbea0067945899d2c8c75b7973660/users/96a87ea8169f4303b153ccfdfdb6a668/roles/b1849c34fc37410098b94ff687ab0055` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:44.656662 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5753f0-3ed2-4f5d-a252-c24030cb1e2f None admin] RBAC: Authorizing `identity:create_grant(project_id=a54cbea0067945899d2c8c75b7973660, user_id=96a87ea8169f4303b153ccfdfdb6a668, role_id=b1849c34fc37410098b94ff687ab0055)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:44.658172 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5753f0-3ed2-4f5d-a252-c24030cb1e2f None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:44.755266 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 116/234] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 21 22:34:44 2022] PUT /identity/v3/projects/a54cbea0067945899d2c8c75b7973660/users/96a87ea8169f4303b153ccfdfdb6a668/roles/b1849c34fc37410098b94ff687ab0055 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:44.775609 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d1c911-e5e9-4953-b79a-788b252ea41f None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:44.775949 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d1c911-e5e9-4953-b79a-788b252ea41f None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:44.776166 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d1c911-e5e9-4953-b79a-788b252ea41f None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:44.777818 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 119/235] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:44.783525 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb692e14-8202-4415-bd38-c26d76567e78 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:44.783802 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb692e14-8202-4415-bd38-c26d76567e78 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:44.784084 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb692e14-8202-4415-bd38-c26d76567e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:44.789468 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:44.803807 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-eb692e14-8202-4415-bd38-c26d76567e78 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:44.805078 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-eb692e14-8202-4415-bd38-c26d76567e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:44.901323 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 117/236] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:34:44.938695 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d1c911-e5e9-4953-b79a-788b252ea41f None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:45.009157 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5049c32-7443-4a07-ac94-3921446b7eee None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:45.009454 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5049c32-7443-4a07-ac94-3921446b7eee None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:45.009791 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5049c32-7443-4a07-ac94-3921446b7eee None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:45.011231 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 118/237] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:45.016451 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf54f8-a63a-460c-9daf-f3296a69a9b9 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:45.016743 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf54f8-a63a-460c-9daf-f3296a69a9b9 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:45.017027 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf54f8-a63a-460c-9daf-f3296a69a9b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:45.038418 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-58cf54f8-a63a-460c-9daf-f3296a69a9b9 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:45.039639 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-58cf54f8-a63a-460c-9daf-f3296a69a9b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:45.085248 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d1c911-e5e9-4953-b79a-788b252ea41f None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:45.115137 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:45.159516 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d334b8-dc89-4929-a436-7d8672e5f4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:45.160456 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d334b8-dc89-4929-a436-7d8672e5f4bd None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:45.160743 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d334b8-dc89-4929-a436-7d8672e5f4bd None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:45.161020 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d334b8-dc89-4929-a436-7d8672e5f4bd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:45.161958 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 119/238] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:34:45.187384 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d334b8-dc89-4929-a436-7d8672e5f4bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:34:45.187905 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cf54f8-a63a-460c-9daf-f3296a69a9b9 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:45.195272 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-a7d334b8-dc89-4929-a436-7d8672e5f4bd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:34:45.195272 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:34:45.195272 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:34:45.195272 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:34:45.195272 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:34:45.195272 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:34:45.195272 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:34:45.195272 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:34:45.195272 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:34:45.195272 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:34:45.195272 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:34:45.195272 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:34:45.195272 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:34:45.195272 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:34:45.195272 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:34:45.195272 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:34:45.196140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:34:45.196140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:34:45.196140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:34:45.196140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:34:45.196140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:34:45.196140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:34:45.196140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:34:45.196140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:34:45.196140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:34:45.196140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:34:45.196140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:34:45.196140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:34:45.196140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:34:45.196140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:34:45.196140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:34:45.196140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:34:45.196922 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:34:45.196922 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:34:45.196922 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:34:45.196922 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:34:45.196922 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:34:45.196922 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:34:45.196922 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:34:45.196922 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:34:45.196922 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:34:45.196922 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:34:45.196922 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:34:45.196922 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:34:45.196922 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 120/239] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:34:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:34:45.203782 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d334b8-dc89-4929-a436-7d8672e5f4bd None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:45.253576 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:45.296304 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cf54f8-a63a-460c-9daf-f3296a69a9b9 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:45.332431 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb21b5dd-efae-471f-9a66-cc78822181f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:45.333236 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21b5dd-efae-471f-9a66-cc78822181f4 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:45.333521 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21b5dd-efae-471f-9a66-cc78822181f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:45.333798 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21b5dd-efae-471f-9a66-cc78822181f4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:45.334998 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb21b5dd-efae-471f-9a66-cc78822181f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:45.336449 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb21b5dd-efae-471f-9a66-cc78822181f4 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:45.339044 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 120/240] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:34:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:45.344451 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb21b5dd-efae-471f-9a66-cc78822181f4 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:45.449530 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d334b8-dc89-4929-a436-7d8672e5f4bd None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:45.450255 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb21b5dd-efae-471f-9a66-cc78822181f4 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:45.481859 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:45.488815 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc60a7-a033-4d89-ae5c-8110549e38fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:45.489766 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc60a7-a033-4d89-ae5c-8110549e38fc None admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:45.490247 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc60a7-a033-4d89-ae5c-8110549e38fc None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:45.490716 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc60a7-a033-4d89-ae5c-8110549e38fc None admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:45.491712 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fc60a7-a033-4d89-ae5c-8110549e38fc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:45.493372 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fc60a7-a033-4d89-ae5c-8110549e38fc None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:45.510990 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c589f118-6bc3-4939-8fb0-1be5bd0da212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:45.512048 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589f118-6bc3-4939-8fb0-1be5bd0da212 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:45.512529 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589f118-6bc3-4939-8fb0-1be5bd0da212 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:45.513195 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589f118-6bc3-4939-8fb0-1be5bd0da212 None admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:45.514218 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c589f118-6bc3-4939-8fb0-1be5bd0da212 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:45.514863 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c589f118-6bc3-4939-8fb0-1be5bd0da212 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:45.516616 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c589f118-6bc3-4939-8fb0-1be5bd0da212 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:45.526994 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 121/241] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 22:34:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:34:45.532071 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c589f118-6bc3-4939-8fb0-1be5bd0da212 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:45.569451 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:45.597665 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 121/242] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 21 22:34:45 2022] POST /identity/v3/users => generated 293 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:45.658571 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c589f118-6bc3-4939-8fb0-1be5bd0da212 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:45.710696 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74333aac-949f-45e2-b9d7-7f40f397aaa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:45.711551 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74333aac-949f-45e2-b9d7-7f40f397aaa1 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:45.711838 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74333aac-949f-45e2-b9d7-7f40f397aaa1 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:45.712116 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74333aac-949f-45e2-b9d7-7f40f397aaa1 None admin] PATH_INFO: `/v3/users/3871a8838ca14d468338211e6f92f570` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:45.731600 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74333aac-949f-45e2-b9d7-7f40f397aaa1 None admin] RBAC: Authorizing `identity:get_user(user_id=3871a8838ca14d468338211e6f92f570)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:45.733101 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74333aac-949f-45e2-b9d7-7f40f397aaa1 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:45.735213 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 122/243] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 22:34:45 2022] GET /identity/v3/users/3871a8838ca14d468338211e6f92f570 => generated 297 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:34:45.740447 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc60a7-a033-4d89-ae5c-8110549e38fc None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:45.882218 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc60a7-a033-4d89-ae5c-8110549e38fc None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:45.919279 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc93a4c6-ced3-4f67-b6e0-31e65ba11ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:45.920240 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93a4c6-ced3-4f67-b6e0-31e65ba11ca6 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:45.920719 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93a4c6-ced3-4f67-b6e0-31e65ba11ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:45.921209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93a4c6-ced3-4f67-b6e0-31e65ba11ca6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:45.978120 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:45.992599 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc93a4c6-ced3-4f67-b6e0-31e65ba11ca6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:45.994957 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc93a4c6-ced3-4f67-b6e0-31e65ba11ca6 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:46.096952 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956adcae-f109-40cd-bc61-d8bdd9b68d0f None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:46.097440 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956adcae-f109-40cd-bc61-d8bdd9b68d0f None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:46.097539 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956adcae-f109-40cd-bc61-d8bdd9b68d0f None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:46.098897 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 123/244] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:46.105901 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68520a2d-563e-435a-836e-00e93cc5401b None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:46.105901 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68520a2d-563e-435a-836e-00e93cc5401b None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:46.105901 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68520a2d-563e-435a-836e-00e93cc5401b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:46.114429 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 122/245] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:34:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:34:46.122408 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc93a4c6-ced3-4f67-b6e0-31e65ba11ca6 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:46.129472 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-68520a2d-563e-435a-836e-00e93cc5401b None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:46.130679 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-68520a2d-563e-435a-836e-00e93cc5401b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:46.261398 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 124/246] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:34:46.264969 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:46.282496 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc93a4c6-ced3-4f67-b6e0-31e65ba11ca6 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:46.309207 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68520a2d-563e-435a-836e-00e93cc5401b None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:46.321054 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64c79b9-b6bd-45df-b9de-5f3cddcd2585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:46.321918 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64c79b9-b6bd-45df-b9de-5f3cddcd2585 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:46.322724 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64c79b9-b6bd-45df-b9de-5f3cddcd2585 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:46.323156 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64c79b9-b6bd-45df-b9de-5f3cddcd2585 None admin] PATH_INFO: `/v3/projects/a54cbea0067945899d2c8c75b7973660` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:46.324357 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64c79b9-b6bd-45df-b9de-5f3cddcd2585 None admin] RBAC: Authorizing `identity:get_project(project_id=a54cbea0067945899d2c8c75b7973660)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:46.325844 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64c79b9-b6bd-45df-b9de-5f3cddcd2585 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:46.327893 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 123/247] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:34:46 2022] GET /identity/v3/projects/a54cbea0067945899d2c8c75b7973660 => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:46.339236 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64c79b9-b6bd-45df-b9de-5f3cddcd2585 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:46.358751 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:46.444938 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68520a2d-563e-435a-836e-00e93cc5401b None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:46.460522 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64c79b9-b6bd-45df-b9de-5f3cddcd2585 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:46.492579 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98a9757-8d25-462c-967d-b536217bad14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:46.493494 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a9757-8d25-462c-967d-b536217bad14 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:46.493778 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a9757-8d25-462c-967d-b536217bad14 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:46.494085 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a9757-8d25-462c-967d-b536217bad14 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:46.498282 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea40e66-ddbc-498d-9db3-c070f2303ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:46.502795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea40e66-ddbc-498d-9db3-c070f2303ff0 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:46.502795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea40e66-ddbc-498d-9db3-c070f2303ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:46.502795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea40e66-ddbc-498d-9db3-c070f2303ff0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:46.502795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea40e66-ddbc-498d-9db3-c070f2303ff0 None admin] RBAC: Adding query filter params (role.id=6ea16f1a323a473e81c1b161134d7512, scope.project.id=a54cbea0067945899d2c8c75b7973660, user.id=3871a8838ca14d468338211e6f92f570) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:46.502795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea40e66-ddbc-498d-9db3-c070f2303ff0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:46.504329 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea40e66-ddbc-498d-9db3-c070f2303ff0 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:46.510793 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 124/248] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 22:34:46 2022] GET /identity/v3/role_assignments?user.id=3871a8838ca14d468338211e6f92f570&scope.project.id=a54cbea0067945899d2c8c75b7973660&role.id=6ea16f1a323a473e81c1b161134d7512 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:34:46.511630 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98a9757-8d25-462c-967d-b536217bad14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:34:46.519713 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-b98a9757-8d25-462c-967d-b536217bad14 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:34:46.519713 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:34:46.519713 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:34:46.519713 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:34:46.519713 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:34:46.519713 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:34:46.519713 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:34:46.519713 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:34:46.519713 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:34:46.519713 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:34:46.519713 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:34:46.519713 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:34:46.519713 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:34:46.519713 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:34:46.519713 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:34:46.519713 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:34:46.520670 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:34:46.520670 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:34:46.520670 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:34:46.520670 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:34:46.520670 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:34:46.520670 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:34:46.520670 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:34:46.520670 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:34:46.520670 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:34:46.520670 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:34:46.520670 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:34:46.520670 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:34:46.520670 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:34:46.520670 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:34:46.520670 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:34:46.520670 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:34:46.521906 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:34:46.521906 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:34:46.521906 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:34:46.521906 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:34:46.521906 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:34:46.521906 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:34:46.521906 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:34:46.521906 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:34:46.521906 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:34:46.521906 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:34:46.521906 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:34:46.521906 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:34:46.521906 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 125/249] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:34:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:34:46.530156 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea40e66-ddbc-498d-9db3-c070f2303ff0 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:46.563678 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:46.669229 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea40e66-ddbc-498d-9db3-c070f2303ff0 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:46.716109 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe366c9-5940-41b0-8d0c-5229bf12143d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:46.717040 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe366c9-5940-41b0-8d0c-5229bf12143d None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:46.717433 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe366c9-5940-41b0-8d0c-5229bf12143d None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:46.717813 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe366c9-5940-41b0-8d0c-5229bf12143d None admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:46.719027 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe366c9-5940-41b0-8d0c-5229bf12143d None admin] RBAC: Adding query filter params (name=member) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:46.719479 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe366c9-5940-41b0-8d0c-5229bf12143d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:46.721578 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe366c9-5940-41b0-8d0c-5229bf12143d None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:46.734262 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 125/250] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 22:34:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:46.740569 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98a9757-8d25-462c-967d-b536217bad14 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:46.875205 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:46.899105 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98a9757-8d25-462c-967d-b536217bad14 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:46.945004 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617a48ee-300e-4248-8999-24bc4b47a077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:46.945897 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617a48ee-300e-4248-8999-24bc4b47a077 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:46.946182 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617a48ee-300e-4248-8999-24bc4b47a077 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:46.946456 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617a48ee-300e-4248-8999-24bc4b47a077 None admin] PATH_INFO: `/v3/users/ded11401a1b9424ba8a649bb8fc842f9` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:46.948250 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617a48ee-300e-4248-8999-24bc4b47a077 None admin] RBAC: Authorizing `identity:get_user(user_id=ded11401a1b9424ba8a649bb8fc842f9)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:46.949711 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617a48ee-300e-4248-8999-24bc4b47a077 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:46.951757 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 126/251] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 22:34:46 2022] GET /identity/v3/users/ded11401a1b9424ba8a649bb8fc842f9 => generated 297 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:46.957626 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe366c9-5940-41b0-8d0c-5229bf12143d None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:46.977635 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a612a8-b90f-4976-9b8f-f51a84f1dac4 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:46.977635 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a612a8-b90f-4976-9b8f-f51a84f1dac4 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:46.977635 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a612a8-b90f-4976-9b8f-f51a84f1dac4 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:46.977635 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 127/252] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:46.982336 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e3ba0-f311-4ce6-803c-609b680d4891 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:46.982630 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e3ba0-f311-4ce6-803c-609b680d4891 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:46.982967 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e3ba0-f311-4ce6-803c-609b680d4891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:47.011823 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-991e3ba0-f311-4ce6-803c-609b680d4891 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:47.013038 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-991e3ba0-f311-4ce6-803c-609b680d4891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:47.118625 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:47.142015 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe366c9-5940-41b0-8d0c-5229bf12143d None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:47.147147 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 128/253] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:34:47.190285 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e055ab-b358-4325-a315-52d0a70a196f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:47.191196 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e055ab-b358-4325-a315-52d0a70a196f None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:47.191482 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e055ab-b358-4325-a315-52d0a70a196f None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:47.191758 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e055ab-b358-4325-a315-52d0a70a196f None admin] PATH_INFO: `/v3/projects/a54cbea0067945899d2c8c75b7973660` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:47.195518 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e055ab-b358-4325-a315-52d0a70a196f None admin] RBAC: Authorizing `identity:get_project(project_id=a54cbea0067945899d2c8c75b7973660)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:47.196967 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e055ab-b358-4325-a315-52d0a70a196f None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:47.199318 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 126/254] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:34:46 2022] GET /identity/v3/projects/a54cbea0067945899d2c8c75b7973660 => generated 302 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:47.207058 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e3ba0-f311-4ce6-803c-609b680d4891 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:47.208559 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e055ab-b358-4325-a315-52d0a70a196f None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:47.403730 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:47.430634 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e055ab-b358-4325-a315-52d0a70a196f None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:47.465984 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e3ba0-f311-4ce6-803c-609b680d4891 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:47.531579 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad36b0c-76e3-4e9c-8954-67e6c3b76af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:47.532858 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad36b0c-76e3-4e9c-8954-67e6c3b76af9 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:47.533425 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad36b0c-76e3-4e9c-8954-67e6c3b76af9 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:47.533944 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad36b0c-76e3-4e9c-8954-67e6c3b76af9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:47.535875 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0193cb-1f34-4035-b90d-8d74c5ae1e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:47.536954 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0193cb-1f34-4035-b90d-8d74c5ae1e4f None admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:47.537322 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0193cb-1f34-4035-b90d-8d74c5ae1e4f None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:47.537661 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0193cb-1f34-4035-b90d-8d74c5ae1e4f None admin] PATH_INFO: `/v3/projects/a54cbea0067945899d2c8c75b7973660/users/ded11401a1b9424ba8a649bb8fc842f9/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:47.542655 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0193cb-1f34-4035-b90d-8d74c5ae1e4f None admin] RBAC: Authorizing `identity:create_grant(project_id=a54cbea0067945899d2c8c75b7973660, user_id=ded11401a1b9424ba8a649bb8fc842f9, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:47.544206 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0193cb-1f34-4035-b90d-8d74c5ae1e4f None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:47.555059 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad36b0c-76e3-4e9c-8954-67e6c3b76af9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:34:47.566032 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-cad36b0c-76e3-4e9c-8954-67e6c3b76af9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:34:47.566032 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:34:47.566032 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:34:47.566032 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:34:47.566032 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:34:47.566032 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:34:47.566032 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:34:47.566032 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:34:47.566032 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:34:47.566032 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:34:47.566032 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:34:47.566032 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:34:47.566032 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:34:47.566032 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:34:47.566032 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:34:47.566032 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:34:47.566954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:34:47.566954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:34:47.566954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:34:47.566954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:34:47.566954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:34:47.566954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:34:47.566954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:34:47.566954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:34:47.566954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:34:47.566954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:34:47.566954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:34:47.566954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:34:47.566954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:34:47.566954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:34:47.566954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:34:47.566954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:34:47.567727 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:34:47.567727 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:34:47.567727 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:34:47.567727 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:34:47.567727 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:34:47.567727 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:34:47.567727 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:34:47.567727 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:34:47.567727 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:34:47.567727 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:34:47.567727 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:34:47.567727 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:34:47.567727 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 127/255] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 22:34:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:34:47.569170 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a4573-1302-4692-95e2-dc814c81a134 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:47.569507 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a4573-1302-4692-95e2-dc814c81a134 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:47.569790 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a4573-1302-4692-95e2-dc814c81a134 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:47.571195 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 128/256] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:47.575712 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a4573-1302-4692-95e2-dc814c81a134 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:47.610977 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:47.625141 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 129/257] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 21 22:34:47 2022] PUT /identity/v3/projects/a54cbea0067945899d2c8c75b7973660/users/ded11401a1b9424ba8a649bb8fc842f9/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:47.634060 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bebaf6-e2d4-4c4b-93a8-7f1ca5a5c556 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:47.635048 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bebaf6-e2d4-4c4b-93a8-7f1ca5a5c556 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:47.635448 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bebaf6-e2d4-4c4b-93a8-7f1ca5a5c556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:47.673087 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-80bebaf6-e2d4-4c4b-93a8-7f1ca5a5c556 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:47.674720 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-80bebaf6-e2d4-4c4b-93a8-7f1ca5a5c556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:47.793504 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a4573-1302-4692-95e2-dc814c81a134 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:47.873825 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e79c5cd-e083-4816-ba98-24c20dfe3186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:47.874521 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e79c5cd-e083-4816-ba98-24c20dfe3186 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:47.874838 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e79c5cd-e083-4816-ba98-24c20dfe3186 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:47.875122 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e79c5cd-e083-4816-ba98-24c20dfe3186 None admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:47.875956 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e79c5cd-e083-4816-ba98-24c20dfe3186 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:47.876270 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e79c5cd-e083-4816-ba98-24c20dfe3186 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:47.876456 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 130/258] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:34:47.877867 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e79c5cd-e083-4816-ba98-24c20dfe3186 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:47.888125 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 129/259] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:34:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:47.894642 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bebaf6-e2d4-4c4b-93a8-7f1ca5a5c556 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:47.915652 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e79c5cd-e083-4816-ba98-24c20dfe3186 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:47.997641 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:48.021568 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:48.051450 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bebaf6-e2d4-4c4b-93a8-7f1ca5a5c556 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:48.068899 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e79c5cd-e083-4816-ba98-24c20dfe3186 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:48.128798 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8831d2f-c43b-4a89-be21-c674459ee74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:48.129006 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9dfb3a-3b77-4668-9feb-0bec310975c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:48.129617 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8831d2f-c43b-4a89-be21-c674459ee74c None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:48.129723 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9dfb3a-3b77-4668-9feb-0bec310975c2 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:48.129905 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8831d2f-c43b-4a89-be21-c674459ee74c None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:48.130005 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9dfb3a-3b77-4668-9feb-0bec310975c2 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:48.130180 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8831d2f-c43b-4a89-be21-c674459ee74c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:48.130274 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9dfb3a-3b77-4668-9feb-0bec310975c2 None admin] PATH_INFO: `/v3/users/96a87ea8169f4303b153ccfdfdb6a668` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:48.132634 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9dfb3a-3b77-4668-9feb-0bec310975c2 None admin] RBAC: Authorizing `identity:get_user(user_id=96a87ea8169f4303b153ccfdfdb6a668)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:48.134091 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9dfb3a-3b77-4668-9feb-0bec310975c2 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:48.136325 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 131/260] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 22:34:47 2022] GET /identity/v3/users/96a87ea8169f4303b153ccfdfdb6a668 => generated 283 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:48.147897 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8831d2f-c43b-4a89-be21-c674459ee74c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:34:48.160061 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-d8831d2f-c43b-4a89-be21-c674459ee74c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:34:48.160061 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:34:48.160061 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:34:48.160061 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:34:48.160061 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:34:48.160061 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:34:48.160061 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:34:48.160061 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:34:48.160061 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:34:48.160061 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:34:48.160061 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:34:48.160061 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:34:48.160061 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:34:48.160061 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:34:48.160061 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:34:48.160061 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:34:48.161268 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:34:48.161268 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:34:48.161268 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:34:48.161268 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:34:48.161268 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:34:48.161268 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:34:48.161268 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:34:48.161268 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:34:48.161268 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:34:48.161268 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:34:48.161268 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:34:48.161268 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:34:48.161268 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:34:48.161268 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:34:48.161268 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:34:48.161268 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:34:48.162242 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:34:48.162242 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:34:48.162242 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:34:48.162242 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:34:48.162242 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:34:48.162242 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:34:48.162242 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:34:48.162242 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:34:48.162242 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:34:48.162242 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:34:48.162242 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:34:48.162242 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:34:48.163030 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9dfb3a-3b77-4668-9feb-0bec310975c2 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:48.164756 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 130/261] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 21 22:34:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:34:48.171699 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8831d2f-c43b-4a89-be21-c674459ee74c None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:48.283123 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9dfb3a-3b77-4668-9feb-0bec310975c2 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:48.335792 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b264b-df7d-4b38-84b5-8b99ce7af128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:48.336459 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b264b-df7d-4b38-84b5-8b99ce7af128 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:48.336746 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b264b-df7d-4b38-84b5-8b99ce7af128 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:48.337025 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b264b-df7d-4b38-84b5-8b99ce7af128 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:48.375200 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:48.397768 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8831d2f-c43b-4a89-be21-c674459ee74c None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:48.473136 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:48.473891 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30b264b-df7d-4b38-84b5-8b99ce7af128 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:48.475414 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30b264b-df7d-4b38-84b5-8b99ce7af128 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:48.504057 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb68616-2810-468b-a044-b221e89d8103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:48.505022 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb68616-2810-468b-a044-b221e89d8103 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:48.505538 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb68616-2810-468b-a044-b221e89d8103 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:48.511151 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb68616-2810-468b-a044-b221e89d8103 None admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:48.512203 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb68616-2810-468b-a044-b221e89d8103 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:48.512722 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb68616-2810-468b-a044-b221e89d8103 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:48.514311 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb68616-2810-468b-a044-b221e89d8103 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:48.531438 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 131/262] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:34:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:48.539862 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb68616-2810-468b-a044-b221e89d8103 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:48.654798 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 132/263] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:34:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:34:48.664132 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b264b-df7d-4b38-84b5-8b99ce7af128 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:48.691841 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb68616-2810-468b-a044-b221e89d8103 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:48.730976 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee7667-583d-412d-a28b-d20b2fd4e87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:48.730976 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee7667-583d-412d-a28b-d20b2fd4e87f None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:48.730976 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee7667-583d-412d-a28b-d20b2fd4e87f None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:48.730976 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee7667-583d-412d-a28b-d20b2fd4e87f None admin] PATH_INFO: `/v3/users/96a87ea8169f4303b153ccfdfdb6a668` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:48.731503 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceee7667-583d-412d-a28b-d20b2fd4e87f None admin] RBAC: Authorizing `identity:get_user(user_id=96a87ea8169f4303b153ccfdfdb6a668)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:48.733025 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceee7667-583d-412d-a28b-d20b2fd4e87f None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:48.734548 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 132/264] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 22:34:48 2022] GET /identity/v3/users/96a87ea8169f4303b153ccfdfdb6a668 => generated 283 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:48.740358 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee7667-583d-412d-a28b-d20b2fd4e87f None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:48.766059 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:48.811792 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:48.851596 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee7667-583d-412d-a28b-d20b2fd4e87f None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:48.887949 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fc7b4a-30cd-437b-9d5e-374bfab8b1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:48.888706 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc7b4a-30cd-437b-9d5e-374bfab8b1d5 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:48.889516 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc7b4a-30cd-437b-9d5e-374bfab8b1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:48.889516 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc7b4a-30cd-437b-9d5e-374bfab8b1d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:48.941946 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b264b-df7d-4b38-84b5-8b99ce7af128 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:48.960522 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fc7b4a-30cd-437b-9d5e-374bfab8b1d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:48.962278 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fc7b4a-30cd-437b-9d5e-374bfab8b1d5 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:49.027155 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:49.069496 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 133/265] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:34:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 22:34:49.074460 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e113513e-90a5-4cbf-87f8-247f81484c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:49.075824 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fc7b4a-30cd-437b-9d5e-374bfab8b1d5 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:49.076329 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113513e-90a5-4cbf-87f8-247f81484c0d None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:49.076942 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113513e-90a5-4cbf-87f8-247f81484c0d None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:49.077544 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113513e-90a5-4cbf-87f8-247f81484c0d None admin] PATH_INFO: `/v3/projects/a54cbea0067945899d2c8c75b7973660` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:49.079568 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e113513e-90a5-4cbf-87f8-247f81484c0d None admin] RBAC: Authorizing `identity:get_project(project_id=a54cbea0067945899d2c8c75b7973660)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:49.081999 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e113513e-90a5-4cbf-87f8-247f81484c0d None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:49.085337 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 133/266] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:34:48 2022] GET /identity/v3/projects/a54cbea0067945899d2c8c75b7973660 => generated 302 bytes in 421 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:49.096167 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e113513e-90a5-4cbf-87f8-247f81484c0d None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:49.313061 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e113513e-90a5-4cbf-87f8-247f81484c0d None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:49.327896 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:49.363004 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fc7b4a-30cd-437b-9d5e-374bfab8b1d5 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:49.372878 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f072f1-929b-48fc-b90a-e7cf14e48856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:49.373556 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f072f1-929b-48fc-b90a-e7cf14e48856 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:49.374020 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f072f1-929b-48fc-b90a-e7cf14e48856 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:49.374311 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f072f1-929b-48fc-b90a-e7cf14e48856 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:49.375210 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f072f1-929b-48fc-b90a-e7cf14e48856 None admin] RBAC: Adding query filter params (role.id=d1ddee590a964ff1bbb4ee8716173021, scope.project.id=a54cbea0067945899d2c8c75b7973660, user.id=96a87ea8169f4303b153ccfdfdb6a668) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:49.375523 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f072f1-929b-48fc-b90a-e7cf14e48856 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:49.376977 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f072f1-929b-48fc-b90a-e7cf14e48856 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:49.388174 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 134/267] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 22:34:49 2022] GET /identity/v3/role_assignments?user.id=96a87ea8169f4303b153ccfdfdb6a668&scope.project.id=a54cbea0067945899d2c8c75b7973660&role.id=d1ddee590a964ff1bbb4ee8716173021 => generated 608 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:34:49.389656 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6924329c-a8e5-4eef-bb67-73c92a033df8 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:49.390083 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6924329c-a8e5-4eef-bb67-73c92a033df8 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:49.390524 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6924329c-a8e5-4eef-bb67-73c92a033df8 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:49.392031 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 135/268] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:49.404584 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef1bc09-e526-4f37-b1b3-2262e87b921b None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:49.404946 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef1bc09-e526-4f37-b1b3-2262e87b921b None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:49.405144 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef1bc09-e526-4f37-b1b3-2262e87b921b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:49.433305 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-eef1bc09-e526-4f37-b1b3-2262e87b921b None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:49.434553 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-eef1bc09-e526-4f37-b1b3-2262e87b921b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:49.455640 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb287e9-4714-4a43-b5f2-38fa5c296560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:49.457015 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb287e9-4714-4a43-b5f2-38fa5c296560 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:49.457835 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb287e9-4714-4a43-b5f2-38fa5c296560 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:49.458277 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb287e9-4714-4a43-b5f2-38fa5c296560 None admin] PATH_INFO: `/v3/projects/a54cbea0067945899d2c8c75b7973660` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:49.459730 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb287e9-4714-4a43-b5f2-38fa5c296560 None admin] RBAC: Authorizing `identity:get_project(project_id=a54cbea0067945899d2c8c75b7973660)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:49.461189 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb287e9-4714-4a43-b5f2-38fa5c296560 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:49.463300 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 134/269] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:34:49 2022] GET /identity/v3/projects/a54cbea0067945899d2c8c75b7973660 => generated 302 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:49.464565 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a28cbb0-e327-49d3-90a6-dce9cc486ca9 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:49.465089 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a28cbb0-e327-49d3-90a6-dce9cc486ca9 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:49.465368 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a28cbb0-e327-49d3-90a6-dce9cc486ca9 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:49.467311 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 135/270] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:49.471945 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a28cbb0-e327-49d3-90a6-dce9cc486ca9 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:49.524417 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:49.621632 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a28cbb0-e327-49d3-90a6-dce9cc486ca9 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:49.639426 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 136/271] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:34:49.640633 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e40bb5-bc16-4e2d-8363-2d896272634a None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:49.640927 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e40bb5-bc16-4e2d-8363-2d896272634a None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:49.641240 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e40bb5-bc16-4e2d-8363-2d896272634a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:49.667333 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-c9e40bb5-bc16-4e2d-8363-2d896272634a None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:49.669983 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-c9e40bb5-bc16-4e2d-8363-2d896272634a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:49.696200 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ec49b7-a5a8-43d8-ba11-36bef464607f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:49.697797 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec49b7-a5a8-43d8-ba11-36bef464607f None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:49.698085 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec49b7-a5a8-43d8-ba11-36bef464607f None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:49.698391 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec49b7-a5a8-43d8-ba11-36bef464607f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:49.699911 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ec49b7-a5a8-43d8-ba11-36bef464607f None admin] RBAC: Adding query filter params (role.id=b1849c34fc37410098b94ff687ab0055, scope.project.id=a54cbea0067945899d2c8c75b7973660, user.id=96a87ea8169f4303b153ccfdfdb6a668) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:49.700230 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ec49b7-a5a8-43d8-ba11-36bef464607f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:49.701710 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ec49b7-a5a8-43d8-ba11-36bef464607f None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:49.710864 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 136/272] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 22:34:49 2022] GET /identity/v3/role_assignments?user.id=96a87ea8169f4303b153ccfdfdb6a668&scope.project.id=a54cbea0067945899d2c8c75b7973660&role.id=b1849c34fc37410098b94ff687ab0055 => generated 608 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:49.712622 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ec49b7-a5a8-43d8-ba11-36bef464607f None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:49.807143 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 137/273] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:34:49.858764 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ec49b7-a5a8-43d8-ba11-36bef464607f None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:49.863240 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:49.893901 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c10726-fbd6-45bc-b1dd-f2d0f0072068 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:49.893901 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c10726-fbd6-45bc-b1dd-f2d0f0072068 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:49.893901 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c10726-fbd6-45bc-b1dd-f2d0f0072068 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:49.894399 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 138/274] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:49.896650 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c10726-fbd6-45bc-b1dd-f2d0f0072068 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:49.904241 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8668c830-d9a7-425c-9deb-cd76fbe4d5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:49.905077 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8668c830-d9a7-425c-9deb-cd76fbe4d5fd None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:49.905378 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8668c830-d9a7-425c-9deb-cd76fbe4d5fd None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:49.905841 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8668c830-d9a7-425c-9deb-cd76fbe4d5fd None admin] PATH_INFO: `/v3/domains/default` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:49.910336 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8668c830-d9a7-425c-9deb-cd76fbe4d5fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:49.913403 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8668c830-d9a7-425c-9deb-cd76fbe4d5fd None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:49.915488 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 137/275] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:34:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:49.917618 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b8da4-81b0-416f-abd4-e8aa4480b89a None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:49.918105 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b8da4-81b0-416f-abd4-e8aa4480b89a None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:49.918575 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b8da4-81b0-416f-abd4-e8aa4480b89a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:49.947630 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-420b8da4-81b0-416f-abd4-e8aa4480b89a None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:49.949172 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-420b8da4-81b0-416f-abd4-e8aa4480b89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:49.952424 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:50.034830 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 138/276] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:34:50.034830 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b8da4-81b0-416f-abd4-e8aa4480b89a None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:50.038743 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c10726-fbd6-45bc-b1dd-f2d0f0072068 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:50.086759 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5c20f4-2252-4a5f-8386-e95057dcbf7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:50.088194 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c20f4-2252-4a5f-8386-e95057dcbf7b None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:50.088474 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c20f4-2252-4a5f-8386-e95057dcbf7b None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:50.088746 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c20f4-2252-4a5f-8386-e95057dcbf7b None admin] PATH_INFO: `/v3/roles/member` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:50.100339 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5c20f4-2252-4a5f-8386-e95057dcbf7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:34:50.108270 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-fe5c20f4-2252-4a5f-8386-e95057dcbf7b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:34:50.108270 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:34:50.108270 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:34:50.108270 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:34:50.108270 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:34:50.108270 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:34:50.108270 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:34:50.108270 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:34:50.108270 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:34:50.108270 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:34:50.108270 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:34:50.108270 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:34:50.108270 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:34:50.108270 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:34:50.108270 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:34:50.108270 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:34:50.109142 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:34:50.109142 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:34:50.109142 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:34:50.109142 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:34:50.109142 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:34:50.109142 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:34:50.109142 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:34:50.109142 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:34:50.109142 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:34:50.109142 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:34:50.109142 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:34:50.109142 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:34:50.109142 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:34:50.109142 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:34:50.109142 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:34:50.109142 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:34:50.110021 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:34:50.110021 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:34:50.110021 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:34:50.110021 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:34:50.110021 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:34:50.110021 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:34:50.110021 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:34:50.110021 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:34:50.110021 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:34:50.110021 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:34:50.110021 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:34:50.110021 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:34:50.110021 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 139/277] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:34:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:34:50.110675 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5c20f4-2252-4a5f-8386-e95057dcbf7b None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:50.148638 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:50.159902 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b8da4-81b0-416f-abd4-e8aa4480b89a None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:50.195980 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ff28d-2f89-41eb-9087-665ff17ad6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:50.196659 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ff28d-2f89-41eb-9087-665ff17ad6b9 None admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:50.197570 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ff28d-2f89-41eb-9087-665ff17ad6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:50.197835 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ff28d-2f89-41eb-9087-665ff17ad6b9 None admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:50.198539 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025ff28d-2f89-41eb-9087-665ff17ad6b9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:50.200035 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025ff28d-2f89-41eb-9087-665ff17ad6b9 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:50.208383 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:50.216760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5c20f4-2252-4a5f-8386-e95057dcbf7b None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:50.250423 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d8a9e0-c944-44e5-a591-13b1d0c35002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:50.251272 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d8a9e0-c944-44e5-a591-13b1d0c35002 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:50.251552 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d8a9e0-c944-44e5-a591-13b1d0c35002 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:50.251824 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d8a9e0-c944-44e5-a591-13b1d0c35002 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:50.262460 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d8a9e0-c944-44e5-a591-13b1d0c35002 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:34:50.270227 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-c4d8a9e0-c944-44e5-a591-13b1d0c35002 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:34:50.270227 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:34:50.270227 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:34:50.270227 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:34:50.270227 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:34:50.270227 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:34:50.270227 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:34:50.270227 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:34:50.270227 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:34:50.270227 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:34:50.270227 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:34:50.270227 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:34:50.270227 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:34:50.270227 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:34:50.270227 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:34:50.270227 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:34:50.271335 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:34:50.271335 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:34:50.271335 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:34:50.271335 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:34:50.271335 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:34:50.271335 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:34:50.271335 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:34:50.271335 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:34:50.271335 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:34:50.271335 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:34:50.271335 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:34:50.271335 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:34:50.271335 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:34:50.271335 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:34:50.271335 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:34:50.271335 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:34:50.272140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:34:50.272140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:34:50.272140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:34:50.272140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:34:50.272140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:34:50.272140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:34:50.272140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:34:50.272140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:34:50.272140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:34:50.272140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:34:50.272140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:34:50.272140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:34:50.272140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 140/278] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:34:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:34:50.275609 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d8a9e0-c944-44e5-a591-13b1d0c35002 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:50.317755 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 139/279] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 21 22:34:50 2022] POST /identity/v3/users => generated 293 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:50.319027 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ff28d-2f89-41eb-9087-665ff17ad6b9 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:50.426475 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d8a9e0-c944-44e5-a591-13b1d0c35002 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:50.466483 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ff28d-2f89-41eb-9087-665ff17ad6b9 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:50.486221 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7f0a01-2558-4645-9c78-8ed359105b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:50.487300 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f0a01-2558-4645-9c78-8ed359105b8b None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:50.488021 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f0a01-2558-4645-9c78-8ed359105b8b None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:50.488440 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f0a01-2558-4645-9c78-8ed359105b8b None admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:50.489366 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7f0a01-2558-4645-9c78-8ed359105b8b None admin] RBAC: Adding query filter params (name=member) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:50.489817 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7f0a01-2558-4645-9c78-8ed359105b8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:50.492634 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7f0a01-2558-4645-9c78-8ed359105b8b None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:50.507132 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 141/280] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 22:34:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:50.510940 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ffb07b-2e1a-432d-b31a-b20f76c7915a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:50.511620 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ffb07b-2e1a-432d-b31a-b20f76c7915a None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:50.511900 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ffb07b-2e1a-432d-b31a-b20f76c7915a None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:50.512174 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ffb07b-2e1a-432d-b31a-b20f76c7915a None admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:50.513376 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7f0a01-2558-4645-9c78-8ed359105b8b None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:50.514213 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ffb07b-2e1a-432d-b31a-b20f76c7915a None admin] RBAC: Adding query filter params (name=reader) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:50.514739 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ffb07b-2e1a-432d-b31a-b20f76c7915a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:50.517247 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ffb07b-2e1a-432d-b31a-b20f76c7915a None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:50.532639 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 140/281] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 22:34:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:50.538941 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:50.539751 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ffb07b-2e1a-432d-b31a-b20f76c7915a None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:50.553733 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:50.644999 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7f0a01-2558-4645-9c78-8ed359105b8b None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:50.685175 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ffb07b-2e1a-432d-b31a-b20f76c7915a None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:50.693826 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b8f651-67e5-41b6-93c5-58ce706a10c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:50.695464 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b8f651-67e5-41b6-93c5-58ce706a10c8 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:50.695958 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b8f651-67e5-41b6-93c5-58ce706a10c8 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:50.696797 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b8f651-67e5-41b6-93c5-58ce706a10c8 None admin] PATH_INFO: `/v3/users/9a7047999f054d4f89e4137b0a2f9994` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:50.716179 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b8f651-67e5-41b6-93c5-58ce706a10c8 None admin] RBAC: Authorizing `identity:get_user(user_id=9a7047999f054d4f89e4137b0a2f9994)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:50.718225 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b8f651-67e5-41b6-93c5-58ce706a10c8 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:50.720599 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 142/282] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 22:34:50 2022] GET /identity/v3/users/9a7047999f054d4f89e4137b0a2f9994 => generated 293 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:34:50.726814 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b8f651-67e5-41b6-93c5-58ce706a10c8 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:50.735350 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4103dab-b902-4ade-bce0-a7486e7ac82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:50.736882 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4103dab-b902-4ade-bce0-a7486e7ac82c None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:50.737240 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4103dab-b902-4ade-bce0-a7486e7ac82c None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:50.737592 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4103dab-b902-4ade-bce0-a7486e7ac82c None admin] PATH_INFO: `/v3/users/3871a8838ca14d468338211e6f92f570` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:50.739705 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4103dab-b902-4ade-bce0-a7486e7ac82c None admin] RBAC: Authorizing `identity:get_user(user_id=3871a8838ca14d468338211e6f92f570)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:50.741881 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4103dab-b902-4ade-bce0-a7486e7ac82c None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:50.744805 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 141/283] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 22:34:50 2022] GET /identity/v3/users/3871a8838ca14d468338211e6f92f570 => generated 297 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:50.753554 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4103dab-b902-4ade-bce0-a7486e7ac82c None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:50.839170 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:50.867196 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b8f651-67e5-41b6-93c5-58ce706a10c8 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:50.894737 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:50.908959 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c6ab32-be3e-4f4e-9fd6-1875f9e25d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:50.909617 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c6ab32-be3e-4f4e-9fd6-1875f9e25d88 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:50.910063 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c6ab32-be3e-4f4e-9fd6-1875f9e25d88 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:50.910345 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c6ab32-be3e-4f4e-9fd6-1875f9e25d88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:50.912303 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c6ab32-be3e-4f4e-9fd6-1875f9e25d88 None admin] RBAC: Adding query filter params (role.id=53d3bdbe9cdf4d35bc73854a33df8e45, user.id=9a7047999f054d4f89e4137b0a2f9994, scope.system=all) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:50.912303 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c6ab32-be3e-4f4e-9fd6-1875f9e25d88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:50.913028 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c6ab32-be3e-4f4e-9fd6-1875f9e25d88 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:50.921047 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 143/284] 10.222.0.52 () {56 vars in 1308 bytes} [Thu Jul 21 22:34:50 2022] GET /identity/v3/role_assignments?user.id=9a7047999f054d4f89e4137b0a2f9994&scope.system=all&role.id=53d3bdbe9cdf4d35bc73854a33df8e45 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:50.979897 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4103dab-b902-4ade-bce0-a7486e7ac82c None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:51.032908 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24623620-34e5-4fdd-aad0-44a443aa04c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:51.033788 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24623620-34e5-4fdd-aad0-44a443aa04c9 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:51.034064 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24623620-34e5-4fdd-aad0-44a443aa04c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:51.034332 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24623620-34e5-4fdd-aad0-44a443aa04c9 None admin] PATH_INFO: `/v3/projects/a54cbea0067945899d2c8c75b7973660` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:51.039741 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24623620-34e5-4fdd-aad0-44a443aa04c9 None admin] RBAC: Authorizing `identity:get_project(project_id=a54cbea0067945899d2c8c75b7973660)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:51.041324 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24623620-34e5-4fdd-aad0-44a443aa04c9 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:51.043357 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 142/285] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:34:50 2022] GET /identity/v3/projects/a54cbea0067945899d2c8c75b7973660 => generated 302 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:51.049611 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c6ab32-be3e-4f4e-9fd6-1875f9e25d88 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:51.171807 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797726ad-13ba-4aba-a1b1-c59f1b5f4f3d None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:51.172191 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797726ad-13ba-4aba-a1b1-c59f1b5f4f3d None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:51.172475 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797726ad-13ba-4aba-a1b1-c59f1b5f4f3d None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:51.174858 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 143/286] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:51.182223 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc33b7-b86a-4a6a-af76-8e5afe76be70 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:51.182511 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc33b7-b86a-4a6a-af76-8e5afe76be70 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:51.182610 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c6ab32-be3e-4f4e-9fd6-1875f9e25d88 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:51.182938 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc33b7-b86a-4a6a-af76-8e5afe76be70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:51.210815 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-e2dc33b7-b86a-4a6a-af76-8e5afe76be70 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:51.212022 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-e2dc33b7-b86a-4a6a-af76-8e5afe76be70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:51.250077 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc116c4-eeae-43bf-9987-e698f3172373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:51.250936 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc116c4-eeae-43bf-9987-e698f3172373 None admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:51.251220 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc116c4-eeae-43bf-9987-e698f3172373 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:51.251500 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc116c4-eeae-43bf-9987-e698f3172373 None admin] PATH_INFO: `/v3/projects/a54cbea0067945899d2c8c75b7973660/users/3871a8838ca14d468338211e6f92f570/roles/6ea16f1a323a473e81c1b161134d7512` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:51.255649 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc116c4-eeae-43bf-9987-e698f3172373 None admin] RBAC: Authorizing `identity:create_grant(project_id=a54cbea0067945899d2c8c75b7973660, user_id=3871a8838ca14d468338211e6f92f570, role_id=6ea16f1a323a473e81c1b161134d7512)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:51.257090 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc116c4-eeae-43bf-9987-e698f3172373 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:51.267110 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:51.316097 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 144/287] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:34:51.347391 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 144/288] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 21 22:34:51 2022] PUT /identity/v3/projects/a54cbea0067945899d2c8c75b7973660/users/3871a8838ca14d468338211e6f92f570/roles/6ea16f1a323a473e81c1b161134d7512 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:51.348405 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dc33b7-b86a-4a6a-af76-8e5afe76be70 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:51.457502 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:51.499347 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dc33b7-b86a-4a6a-af76-8e5afe76be70 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:51.544719 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e76293e-a9b4-46ed-ac03-22b1d5117610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:51.545472 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76293e-a9b4-46ed-ac03-22b1d5117610 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:51.545789 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76293e-a9b4-46ed-ac03-22b1d5117610 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:51.546065 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76293e-a9b4-46ed-ac03-22b1d5117610 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:51.562012 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e76293e-a9b4-46ed-ac03-22b1d5117610 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:34:51.572277 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-3e76293e-a9b4-46ed-ac03-22b1d5117610 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:34:51.572277 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:34:51.572277 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:34:51.572277 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:34:51.572277 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:34:51.572277 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:34:51.572277 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:34:51.572277 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:34:51.572277 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:34:51.572277 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:34:51.572277 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:34:51.572277 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:34:51.572277 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:34:51.572277 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:34:51.572277 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:34:51.572277 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:34:51.573190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:34:51.573190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:34:51.573190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:34:51.573190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:34:51.573190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:34:51.573190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:34:51.573190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:34:51.573190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:34:51.573190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:34:51.573190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:34:51.573190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:34:51.573190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:34:51.573190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:34:51.573190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:34:51.573190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:34:51.573190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:34:51.573980 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:34:51.573980 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:34:51.573980 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:34:51.573980 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:34:51.573980 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:34:51.573980 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:34:51.573980 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:34:51.573980 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:34:51.573980 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:34:51.573980 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:34:51.573980 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:34:51.573980 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:34:51.573980 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 145/289] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:34:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:34:51.579858 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc116c4-eeae-43bf-9987-e698f3172373 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:51.685929 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc116c4-eeae-43bf-9987-e698f3172373 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:51.720547 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087dc649-08a0-47bd-9771-af91d6bc7fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:51.721216 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087dc649-08a0-47bd-9771-af91d6bc7fa2 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:51.721501 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087dc649-08a0-47bd-9771-af91d6bc7fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:51.721778 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087dc649-08a0-47bd-9771-af91d6bc7fa2 None admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:51.722528 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087dc649-08a0-47bd-9771-af91d6bc7fa2 None admin] RBAC: Adding query filter params (name=member) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:51.722859 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087dc649-08a0-47bd-9771-af91d6bc7fa2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:51.724233 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087dc649-08a0-47bd-9771-af91d6bc7fa2 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:51.733233 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 145/290] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 22:34:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:51.738693 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e76293e-a9b4-46ed-ac03-22b1d5117610 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:51.890347 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e76293e-a9b4-46ed-ac03-22b1d5117610 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:51.939629 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8ed4b3-c2d6-4510-b608-718bbfbf862e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:51.940540 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ed4b3-c2d6-4510-b608-718bbfbf862e None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:51.940823 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ed4b3-c2d6-4510-b608-718bbfbf862e None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:51.941099 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ed4b3-c2d6-4510-b608-718bbfbf862e None admin] PATH_INFO: `/v3/users/ded11401a1b9424ba8a649bb8fc842f9` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:51.948127 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8ed4b3-c2d6-4510-b608-718bbfbf862e None admin] RBAC: Authorizing `identity:get_user(user_id=ded11401a1b9424ba8a649bb8fc842f9)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:51.949604 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8ed4b3-c2d6-4510-b608-718bbfbf862e None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:51.951642 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 146/291] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 22:34:51 2022] GET /identity/v3/users/ded11401a1b9424ba8a649bb8fc842f9 => generated 297 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:51.957018 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087dc649-08a0-47bd-9771-af91d6bc7fa2 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:52.039427 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:52.075599 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087dc649-08a0-47bd-9771-af91d6bc7fa2 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:52.111592 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848a0359-fce3-4185-9d4c-8270e263367a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:52.112253 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a0359-fce3-4185-9d4c-8270e263367a None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:52.112538 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a0359-fce3-4185-9d4c-8270e263367a None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:52.112813 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a0359-fce3-4185-9d4c-8270e263367a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:52.186888 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848a0359-fce3-4185-9d4c-8270e263367a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:52.188286 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848a0359-fce3-4185-9d4c-8270e263367a None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:52.259135 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:52.316229 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 146/292] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:34:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:34:52.322420 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8ed4b3-c2d6-4510-b608-718bbfbf862e None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:52.325599 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:52.455784 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8ed4b3-c2d6-4510-b608-718bbfbf862e None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:52.494320 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b3489-d53f-4e10-b989-f2640dcc7c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:52.495227 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b3489-d53f-4e10-b989-f2640dcc7c4c None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:52.495519 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b3489-d53f-4e10-b989-f2640dcc7c4c None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:52.495792 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b3489-d53f-4e10-b989-f2640dcc7c4c None admin] PATH_INFO: `/v3/projects/a54cbea0067945899d2c8c75b7973660` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:52.496991 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65b3489-d53f-4e10-b989-f2640dcc7c4c None admin] RBAC: Authorizing `identity:get_project(project_id=a54cbea0067945899d2c8c75b7973660)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:52.498436 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65b3489-d53f-4e10-b989-f2640dcc7c4c None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:52.500494 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 147/293] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:34:52 2022] GET /identity/v3/projects/a54cbea0067945899d2c8c75b7973660 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:52.506472 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848a0359-fce3-4185-9d4c-8270e263367a None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:52.613049 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848a0359-fce3-4185-9d4c-8270e263367a None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:52.647627 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb0fbf-b266-473c-a009-bdd5a19377fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:52.648349 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb0fbf-b266-473c-a009-bdd5a19377fe None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:52.648632 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb0fbf-b266-473c-a009-bdd5a19377fe None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:52.648911 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb0fbf-b266-473c-a009-bdd5a19377fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:52.650413 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bb0fbf-b266-473c-a009-bdd5a19377fe None admin] RBAC: Adding query filter params (role.id=53d3bdbe9cdf4d35bc73854a33df8e45, scope.project.id=a54cbea0067945899d2c8c75b7973660, user.id=ded11401a1b9424ba8a649bb8fc842f9) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:52.650757 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bb0fbf-b266-473c-a009-bdd5a19377fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:52.652233 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bb0fbf-b266-473c-a009-bdd5a19377fe None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:52.658742 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 147/294] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 22:34:52 2022] GET /identity/v3/role_assignments?user.id=ded11401a1b9424ba8a649bb8fc842f9&scope.project.id=a54cbea0067945899d2c8c75b7973660&role.id=53d3bdbe9cdf4d35bc73854a33df8e45 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:53.111187 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe52c77-cf8a-4870-9382-fbb53a5f0813 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:53.111696 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe52c77-cf8a-4870-9382-fbb53a5f0813 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:53.111799 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe52c77-cf8a-4870-9382-fbb53a5f0813 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:53.113408 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 148/295] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:53.124851 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79dfc45-25de-44e7-9b00-7f72cf62bb2e None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:53.125449 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79dfc45-25de-44e7-9b00-7f72cf62bb2e None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:53.125968 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79dfc45-25de-44e7-9b00-7f72cf62bb2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:53.146725 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-f79dfc45-25de-44e7-9b00-7f72cf62bb2e None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:53.148232 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-f79dfc45-25de-44e7-9b00-7f72cf62bb2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:53.171933 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:53.235336 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 148/296] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:34:53.260805 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe52c77-cf8a-4870-9382-fbb53a5f0813 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:53.265760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6513bb22-649c-4000-9f4c-8c5a5d577837 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:53.266276 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6513bb22-649c-4000-9f4c-8c5a5d577837 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:53.266754 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6513bb22-649c-4000-9f4c-8c5a5d577837 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:53.268599 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 149/297] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:53.276262 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95872542-6d6d-43bd-bcf7-b822ce69ed49 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:53.276780 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95872542-6d6d-43bd-bcf7-b822ce69ed49 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:53.277252 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95872542-6d6d-43bd-bcf7-b822ce69ed49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:53.297064 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-95872542-6d6d-43bd-bcf7-b822ce69ed49 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:53.298200 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-95872542-6d6d-43bd-bcf7-b822ce69ed49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:53.337969 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:53.372128 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe52c77-cf8a-4870-9382-fbb53a5f0813 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:53.378812 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 150/298] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:34:53.404210 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95872542-6d6d-43bd-bcf7-b822ce69ed49 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:53.420169 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f0e39e-4086-4fbb-8f6c-a828c2b34ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:53.421192 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f0e39e-4086-4fbb-8f6c-a828c2b34ee1 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:53.421638 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f0e39e-4086-4fbb-8f6c-a828c2b34ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:53.422073 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f0e39e-4086-4fbb-8f6c-a828c2b34ee1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:53.423279 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f0e39e-4086-4fbb-8f6c-a828c2b34ee1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:53.424856 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f0e39e-4086-4fbb-8f6c-a828c2b34ee1 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:53.426620 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 149/299] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:34:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:34:53.434900 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f0e39e-4086-4fbb-8f6c-a828c2b34ee1 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:53.515054 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:53.534074 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95872542-6d6d-43bd-bcf7-b822ce69ed49 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:53.550180 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f0e39e-4086-4fbb-8f6c-a828c2b34ee1 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:53.569715 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c3d982-5fbc-4d51-9b9a-4348db9c8b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:53.570542 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c3d982-5fbc-4d51-9b9a-4348db9c8b5c None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:53.570861 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c3d982-5fbc-4d51-9b9a-4348db9c8b5c None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:53.571140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c3d982-5fbc-4d51-9b9a-4348db9c8b5c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:53.581453 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c3d982-5fbc-4d51-9b9a-4348db9c8b5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:34:53.589052 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-d1c3d982-5fbc-4d51-9b9a-4348db9c8b5c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:34:53.589052 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:34:53.589052 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:34:53.589052 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:34:53.589052 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:34:53.589052 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:34:53.589052 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:34:53.589052 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:34:53.589052 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:34:53.589052 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:34:53.589052 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:34:53.589052 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:34:53.589052 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:34:53.589052 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:34:53.589052 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:34:53.589052 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:34:53.589958 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:34:53.589958 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:34:53.589958 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:34:53.589958 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:34:53.589958 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:34:53.589958 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:34:53.589958 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:34:53.589958 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:34:53.589958 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:34:53.589958 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:34:53.589958 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:34:53.589958 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:34:53.589958 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:34:53.589958 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:34:53.589958 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:34:53.589958 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:34:53.595857 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:34:53.595857 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:34:53.595857 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:34:53.595857 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:34:53.595857 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:34:53.595857 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:34:53.595857 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:34:53.595857 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:34:53.595857 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:34:53.595857 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:34:53.595857 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:34:53.595857 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:34:53.595857 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 151/300] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:34:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:34:53.595857 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf6b22-d6bc-4be3-81f1-48fb2c244f5e None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:53.596566 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf6b22-d6bc-4be3-81f1-48fb2c244f5e None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:53.596566 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf6b22-d6bc-4be3-81f1-48fb2c244f5e None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:53.596566 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 152/301] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:53.596566 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cf6b22-d6bc-4be3-81f1-48fb2c244f5e None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:53.633106 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd0bb2b-f379-4ae9-9dc8-7e512a507c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:53.633813 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd0bb2b-f379-4ae9-9dc8-7e512a507c30 None admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:53.634160 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd0bb2b-f379-4ae9-9dc8-7e512a507c30 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:53.634443 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd0bb2b-f379-4ae9-9dc8-7e512a507c30 None admin] PATH_INFO: `/v3/groups` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:53.635491 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd0bb2b-f379-4ae9-9dc8-7e512a507c30 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:53.636581 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd0bb2b-f379-4ae9-9dc8-7e512a507c30 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:53.699436 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cf6b22-d6bc-4be3-81f1-48fb2c244f5e None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:53.732475 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b10fe7f-9166-48b8-b85f-69ceba8f44a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:53.733136 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b10fe7f-9166-48b8-b85f-69ceba8f44a8 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:53.733422 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b10fe7f-9166-48b8-b85f-69ceba8f44a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:53.733701 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b10fe7f-9166-48b8-b85f-69ceba8f44a8 None admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:53.734449 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b10fe7f-9166-48b8-b85f-69ceba8f44a8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:53.734752 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b10fe7f-9166-48b8-b85f-69ceba8f44a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:53.736143 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b10fe7f-9166-48b8-b85f-69ceba8f44a8 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:53.737511 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:53.746281 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 153/302] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 22:34:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:34:53.747763 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f9b54-95a7-454d-a006-f8d854a16f52 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:53.748178 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f9b54-95a7-454d-a006-f8d854a16f52 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:53.748596 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f9b54-95a7-454d-a006-f8d854a16f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:53.751443 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 150/303] 10.222.0.52 () {60 vars in 1134 bytes} [Thu Jul 21 22:34:53 2022] POST /identity/v3/groups => generated 229 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:53.752387 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd0bb2b-f379-4ae9-9dc8-7e512a507c30 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:53.759845 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:53.772024 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-051f9b54-95a7-454d-a006-f8d854a16f52 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:53.773306 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-051f9b54-95a7-454d-a006-f8d854a16f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:53.865940 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 154/304] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:34:53.889559 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f9b54-95a7-454d-a006-f8d854a16f52 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:53.896955 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd0bb2b-f379-4ae9-9dc8-7e512a507c30 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:53.938437 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e736f2fb-e8b7-47dc-a186-5f6a0b9dd719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:53.939362 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e736f2fb-e8b7-47dc-a186-5f6a0b9dd719 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:53.939640 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e736f2fb-e8b7-47dc-a186-5f6a0b9dd719 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:53.939912 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e736f2fb-e8b7-47dc-a186-5f6a0b9dd719 None admin] PATH_INFO: `/v3/users/835e88eebc8c4731afef79c594c6413c` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:53.956121 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e736f2fb-e8b7-47dc-a186-5f6a0b9dd719 None admin] RBAC: Authorizing `identity:get_user(user_id=835e88eebc8c4731afef79c594c6413c)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:53.957164 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e736f2fb-e8b7-47dc-a186-5f6a0b9dd719 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:53.959081 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 151/305] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 22:34:53 2022] GET /identity/v3/users/835e88eebc8c4731afef79c594c6413c => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:53.960437 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dacc937-381a-4077-ba57-f033d254391f None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:53.960551 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dacc937-381a-4077-ba57-f033d254391f None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:53.960756 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dacc937-381a-4077-ba57-f033d254391f None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:53.962433 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 152/306] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:53.965323 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dacc937-381a-4077-ba57-f033d254391f None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:54.006860 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f9b54-95a7-454d-a006-f8d854a16f52 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:54.036333 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:54.043454 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3ca49c-8fc6-4ca1-9c82-d481d044ba96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:54.044371 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ca49c-8fc6-4ca1-9c82-d481d044ba96 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:54.044583 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ca49c-8fc6-4ca1-9c82-d481d044ba96 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:54.048796 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ca49c-8fc6-4ca1-9c82-d481d044ba96 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:54.048796 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:54.056975 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3ca49c-8fc6-4ca1-9c82-d481d044ba96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:34:54.064741 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-3c3ca49c-8fc6-4ca1-9c82-d481d044ba96 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:34:54.064741 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:34:54.064741 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:34:54.064741 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:34:54.064741 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:34:54.064741 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:34:54.064741 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:34:54.064741 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:34:54.064741 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:34:54.064741 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:34:54.064741 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:34:54.064741 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:34:54.064741 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:34:54.064741 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:34:54.064741 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:34:54.064741 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:34:54.065625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:34:54.065625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:34:54.065625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:34:54.065625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:34:54.065625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:34:54.065625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:34:54.065625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:34:54.065625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:34:54.065625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:34:54.065625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:34:54.065625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:34:54.065625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:34:54.065625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:34:54.065625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:34:54.065625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:34:54.065625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:34:54.066440 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:34:54.066440 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:34:54.066440 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:34:54.066440 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:34:54.066440 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:34:54.066440 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:34:54.066440 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:34:54.066440 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:34:54.066440 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:34:54.066440 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:34:54.066440 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:34:54.066440 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:34:54.070879 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 155/307] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:34:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:34:54.072726 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcdf63d-efdf-45f8-900c-76eb0b6482a4 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:54.073061 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcdf63d-efdf-45f8-900c-76eb0b6482a4 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:54.073340 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcdf63d-efdf-45f8-900c-76eb0b6482a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:54.086327 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dacc937-381a-4077-ba57-f033d254391f None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:54.089513 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-afcdf63d-efdf-45f8-900c-76eb0b6482a4 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:54.090683 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-afcdf63d-efdf-45f8-900c-76eb0b6482a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:54.123755 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2539212f-aa93-4858-b657-7651175be3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:54.124502 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539212f-aa93-4858-b657-7651175be3bb None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:54.124688 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539212f-aa93-4858-b657-7651175be3bb None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:54.124979 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539212f-aa93-4858-b657-7651175be3bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:54.125782 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2539212f-aa93-4858-b657-7651175be3bb None admin] RBAC: Adding query filter params (role.id=6ea16f1a323a473e81c1b161134d7512, user.id=835e88eebc8c4731afef79c594c6413c, scope.system=all) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:54.126088 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2539212f-aa93-4858-b657-7651175be3bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:54.127555 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2539212f-aa93-4858-b657-7651175be3bb None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:54.132790 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 153/308] 10.222.0.52 () {56 vars in 1308 bytes} [Thu Jul 21 22:34:53 2022] GET /identity/v3/role_assignments?user.id=835e88eebc8c4731afef79c594c6413c&scope.system=all&role.id=6ea16f1a323a473e81c1b161134d7512 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:34:54.133490 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2539212f-aa93-4858-b657-7651175be3bb None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:54.169424 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 156/309] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:34:54.194861 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcdf63d-efdf-45f8-900c-76eb0b6482a4 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:54.293516 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2539212f-aa93-4858-b657-7651175be3bb None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:54.313221 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcdf63d-efdf-45f8-900c-76eb0b6482a4 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:54.315050 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:54.330011 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aceda3-b4cc-4887-be90-84df6f770549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:54.330691 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aceda3-b4cc-4887-be90-84df6f770549 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:54.331010 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aceda3-b4cc-4887-be90-84df6f770549 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:54.331290 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aceda3-b4cc-4887-be90-84df6f770549 None admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:54.332046 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aceda3-b4cc-4887-be90-84df6f770549 None admin] RBAC: Adding query filter params (name=member) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:54.334891 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aceda3-b4cc-4887-be90-84df6f770549 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:54.334891 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aceda3-b4cc-4887-be90-84df6f770549 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:54.339125 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:54.346981 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 154/310] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 22:34:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:54.349433 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aceda3-b4cc-4887-be90-84df6f770549 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:54.350303 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd52810-31d5-4992-95e9-d88ad3d7bcee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:54.351189 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd52810-31d5-4992-95e9-d88ad3d7bcee None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:54.351467 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd52810-31d5-4992-95e9-d88ad3d7bcee None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:54.351737 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd52810-31d5-4992-95e9-d88ad3d7bcee None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:54.363624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd52810-31d5-4992-95e9-d88ad3d7bcee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:34:54.371501 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-7fd52810-31d5-4992-95e9-d88ad3d7bcee None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:34:54.371501 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:34:54.371501 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:34:54.371501 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:34:54.371501 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:34:54.371501 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:34:54.371501 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:34:54.371501 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:34:54.371501 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:34:54.371501 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:34:54.371501 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:34:54.371501 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:34:54.371501 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:34:54.371501 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:34:54.371501 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:34:54.371501 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:34:54.372356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:34:54.372356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:34:54.372356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:34:54.372356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:34:54.372356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:34:54.372356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:34:54.372356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:34:54.372356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:34:54.372356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:34:54.372356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:34:54.372356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:34:54.372356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:34:54.372356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:34:54.372356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:34:54.372356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:34:54.372356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:34:54.373142 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:34:54.373142 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:34:54.373142 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:34:54.373142 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:34:54.373142 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:34:54.373142 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:34:54.373142 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:34:54.373142 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:34:54.373142 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:34:54.373142 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:34:54.373142 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:34:54.373142 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:34:54.373142 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 157/311] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:34:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:34:54.379051 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd52810-31d5-4992-95e9-d88ad3d7bcee None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:54.458660 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aceda3-b4cc-4887-be90-84df6f770549 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:54.486796 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd52810-31d5-4992-95e9-d88ad3d7bcee None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:54.507441 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045e9d08-541a-4907-8b00-7d31021823fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:54.508619 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045e9d08-541a-4907-8b00-7d31021823fd None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:54.508920 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045e9d08-541a-4907-8b00-7d31021823fd None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:54.509776 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045e9d08-541a-4907-8b00-7d31021823fd None admin] PATH_INFO: `/v3/users/9a7047999f054d4f89e4137b0a2f9994` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:54.511711 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045e9d08-541a-4907-8b00-7d31021823fd None admin] RBAC: Authorizing `identity:get_user(user_id=9a7047999f054d4f89e4137b0a2f9994)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:54.513989 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045e9d08-541a-4907-8b00-7d31021823fd None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:54.518225 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f1f68-8fc0-427e-97b4-e17a29f076a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:54.519006 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f1f68-8fc0-427e-97b4-e17a29f076a3 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:54.519355 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f1f68-8fc0-427e-97b4-e17a29f076a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:54.519700 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299f1f68-8fc0-427e-97b4-e17a29f076a3 None admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:54.520519 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299f1f68-8fc0-427e-97b4-e17a29f076a3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:54.521221 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 155/312] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 22:34:54 2022] GET /identity/v3/users/9a7047999f054d4f89e4137b0a2f9994 => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:34:54.521327 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299f1f68-8fc0-427e-97b4-e17a29f076a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:54.522102 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045e9d08-541a-4907-8b00-7d31021823fd None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:54.522872 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299f1f68-8fc0-427e-97b4-e17a29f076a3 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:54.533180 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 158/313] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 22:34:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:54.539744 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f1f68-8fc0-427e-97b4-e17a29f076a3 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:54.598614 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:54.599992 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:54.659685 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045e9d08-541a-4907-8b00-7d31021823fd None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:54.673943 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299f1f68-8fc0-427e-97b4-e17a29f076a3 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:54.698687 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618b8a2f-686f-4894-9c88-5edab6fb0d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:54.699870 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b8a2f-686f-4894-9c88-5edab6fb0d8b None admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:54.700334 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b8a2f-686f-4894-9c88-5edab6fb0d8b None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:54.700800 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b8a2f-686f-4894-9c88-5edab6fb0d8b None admin] PATH_INFO: `/v3/system/users/9a7047999f054d4f89e4137b0a2f9994/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:54.702696 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618b8a2f-686f-4894-9c88-5edab6fb0d8b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=9a7047999f054d4f89e4137b0a2f9994, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:54.704276 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618b8a2f-686f-4894-9c88-5edab6fb0d8b None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:54.713638 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba6b29-f5f1-460b-bbb6-e8f62ae344d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:54.714453 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba6b29-f5f1-460b-bbb6-e8f62ae344d4 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:54.714732 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba6b29-f5f1-460b-bbb6-e8f62ae344d4 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:54.715137 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba6b29-f5f1-460b-bbb6-e8f62ae344d4 None admin] PATH_INFO: `/v3/users/3871a8838ca14d468338211e6f92f570` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:54.716316 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ba6b29-f5f1-460b-bbb6-e8f62ae344d4 None admin] RBAC: Authorizing `identity:get_user(user_id=3871a8838ca14d468338211e6f92f570)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:54.717722 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ba6b29-f5f1-460b-bbb6-e8f62ae344d4 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:54.722810 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 159/314] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 22:34:54 2022] GET /identity/v3/users/3871a8838ca14d468338211e6f92f570 => generated 297 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:54.729937 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba6b29-f5f1-460b-bbb6-e8f62ae344d4 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:54.804365 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 156/315] 10.222.0.52 () {58 vars in 1334 bytes} [Thu Jul 21 22:34:54 2022] PUT /identity/v3/system/users/9a7047999f054d4f89e4137b0a2f9994/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:54.834124 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba6b29-f5f1-460b-bbb6-e8f62ae344d4 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:54.851194 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:54.874902 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224b2c1c-db06-4c62-a236-68c54978e930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:54.875573 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224b2c1c-db06-4c62-a236-68c54978e930 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:54.875917 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224b2c1c-db06-4c62-a236-68c54978e930 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:54.876193 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224b2c1c-db06-4c62-a236-68c54978e930 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:54.954101 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224b2c1c-db06-4c62-a236-68c54978e930 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:54.955620 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224b2c1c-db06-4c62-a236-68c54978e930 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:55.070383 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 160/316] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:34:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 22:34:55.076357 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618b8a2f-686f-4894-9c88-5edab6fb0d8b None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:55.181153 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618b8a2f-686f-4894-9c88-5edab6fb0d8b None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:55.194878 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:55.217035 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73702279-695d-4e9d-b6d1-d24fc6f071c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:55.223159 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73702279-695d-4e9d-b6d1-d24fc6f071c9 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:55.223159 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73702279-695d-4e9d-b6d1-d24fc6f071c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:55.223159 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73702279-695d-4e9d-b6d1-d24fc6f071c9 None admin] PATH_INFO: `/v3/projects/a54cbea0067945899d2c8c75b7973660` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:55.223159 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73702279-695d-4e9d-b6d1-d24fc6f071c9 None admin] RBAC: Authorizing `identity:get_project(project_id=a54cbea0067945899d2c8c75b7973660)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:55.223159 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73702279-695d-4e9d-b6d1-d24fc6f071c9 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:55.223159 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 157/317] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:34:55 2022] GET /identity/v3/projects/a54cbea0067945899d2c8c75b7973660 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:55.229755 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224b2c1c-db06-4c62-a236-68c54978e930 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:55.246720 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:55.344115 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224b2c1c-db06-4c62-a236-68c54978e930 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:55.381537 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3be3f3-fe3a-4241-8509-3d42a18c13f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:55.382607 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3be3f3-fe3a-4241-8509-3d42a18c13f7 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:55.383257 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3be3f3-fe3a-4241-8509-3d42a18c13f7 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:55.383733 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3be3f3-fe3a-4241-8509-3d42a18c13f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:55.385283 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3be3f3-fe3a-4241-8509-3d42a18c13f7 None admin] RBAC: Adding query filter params (role.id=6ea16f1a323a473e81c1b161134d7512, scope.project.id=a54cbea0067945899d2c8c75b7973660, user.id=3871a8838ca14d468338211e6f92f570) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:55.385801 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3be3f3-fe3a-4241-8509-3d42a18c13f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:55.387488 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3be3f3-fe3a-4241-8509-3d42a18c13f7 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:55.394143 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 161/318] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 22:34:55 2022] GET /identity/v3/role_assignments?user.id=3871a8838ca14d468338211e6f92f570&scope.project.id=a54cbea0067945899d2c8c75b7973660&role.id=6ea16f1a323a473e81c1b161134d7512 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:55.928174 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded2a8e-f592-4557-a5b9-799f7d3cca62 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:55.928459 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded2a8e-f592-4557-a5b9-799f7d3cca62 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:55.928734 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded2a8e-f592-4557-a5b9-799f7d3cca62 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:55.930118 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 158/319] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:55.936000 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38dde03-aca3-4cae-bf84-d9252e272132 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:55.936418 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38dde03-aca3-4cae-bf84-d9252e272132 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:55.936582 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38dde03-aca3-4cae-bf84-d9252e272132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:55.953754 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-d38dde03-aca3-4cae-bf84-d9252e272132 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:55.954976 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-d38dde03-aca3-4cae-bf84-d9252e272132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:56.030806 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 162/320] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:34:56.049841 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eded2a8e-f592-4557-a5b9-799f7d3cca62 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:56.155365 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eded2a8e-f592-4557-a5b9-799f7d3cca62 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:56.189123 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f121668-037b-47a8-ab23-b44d4ef41da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:56.189848 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f121668-037b-47a8-ab23-b44d4ef41da0 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:56.190132 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f121668-037b-47a8-ab23-b44d4ef41da0 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:56.190407 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f121668-037b-47a8-ab23-b44d4ef41da0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:56.191598 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f121668-037b-47a8-ab23-b44d4ef41da0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:56.193041 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f121668-037b-47a8-ab23-b44d4ef41da0 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:56.194849 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 159/321] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:34:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:34:56.202463 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38dde03-aca3-4cae-bf84-d9252e272132 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:56.203824 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e7963-7a13-4b6e-8031-cbc295db1b7d None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:56.203824 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e7963-7a13-4b6e-8031-cbc295db1b7d None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:56.203993 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e7963-7a13-4b6e-8031-cbc295db1b7d None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:56.205430 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 160/322] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:56.210017 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5633744-5db5-4a0f-9985-f1b969478a00 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:56.210308 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5633744-5db5-4a0f-9985-f1b969478a00 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:56.210587 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5633744-5db5-4a0f-9985-f1b969478a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:56.214726 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:56.228601 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-f5633744-5db5-4a0f-9985-f1b969478a00 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:56.229788 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-f5633744-5db5-4a0f-9985-f1b969478a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:56.277177 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:56.298827 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 161/323] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:34:56.309545 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38dde03-aca3-4cae-bf84-d9252e272132 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:56.319981 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5633744-5db5-4a0f-9985-f1b969478a00 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:56.346391 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b1ff7c-182b-4b9d-9936-b57b378be519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:56.347107 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b1ff7c-182b-4b9d-9936-b57b378be519 None admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:56.347394 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b1ff7c-182b-4b9d-9936-b57b378be519 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:56.347672 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b1ff7c-182b-4b9d-9936-b57b378be519 None admin] PATH_INFO: `/v3/groups` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:56.348371 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b1ff7c-182b-4b9d-9936-b57b378be519 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:56.349827 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b1ff7c-182b-4b9d-9936-b57b378be519 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:56.421228 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5633744-5db5-4a0f-9985-f1b969478a00 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:56.455952 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecae9c6-72ff-48b6-8f6c-44b2db8bc0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:56.456677 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecae9c6-72ff-48b6-8f6c-44b2db8bc0e2 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:56.456957 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecae9c6-72ff-48b6-8f6c-44b2db8bc0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:56.457232 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecae9c6-72ff-48b6-8f6c-44b2db8bc0e2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:56.463068 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 163/324] 10.222.0.52 () {60 vars in 1134 bytes} [Thu Jul 21 22:34:56 2022] POST /identity/v3/groups => generated 226 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:34:56.473326 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecae9c6-72ff-48b6-8f6c-44b2db8bc0e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:34:56.482341 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-5ecae9c6-72ff-48b6-8f6c-44b2db8bc0e2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:34:56.482341 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:34:56.482341 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:34:56.482341 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:34:56.482341 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:34:56.482341 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:34:56.482341 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:34:56.482341 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:34:56.482341 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:34:56.482341 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:34:56.482341 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:34:56.482341 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:34:56.482341 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:34:56.482341 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:34:56.482341 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:34:56.482341 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:34:56.483580 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:34:56.483580 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:34:56.483580 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:34:56.483580 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:34:56.483580 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:34:56.483580 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:34:56.483580 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:34:56.483580 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:34:56.483580 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:34:56.483580 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:34:56.483580 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:34:56.483580 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:34:56.483580 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:34:56.483580 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:34:56.483580 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:34:56.483580 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:34:56.484361 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:34:56.484361 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:34:56.484361 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:34:56.484361 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:34:56.484361 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:34:56.484361 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:34:56.484361 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:34:56.484361 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:34:56.484361 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:34:56.484361 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:34:56.484361 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:34:56.484361 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:34:56.486964 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 162/325] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:34:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:34:56.489877 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b1ff7c-182b-4b9d-9936-b57b378be519 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:56.568450 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:56.606620 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b1ff7c-182b-4b9d-9936-b57b378be519 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:56.658679 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f054a049-5747-4edc-9e90-aac79da298fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:56.659399 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f054a049-5747-4edc-9e90-aac79da298fb None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:56.659687 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f054a049-5747-4edc-9e90-aac79da298fb None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:56.659965 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f054a049-5747-4edc-9e90-aac79da298fb None admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:56.660724 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f054a049-5747-4edc-9e90-aac79da298fb None admin] RBAC: Adding query filter params (name=reader) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:56.661039 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f054a049-5747-4edc-9e90-aac79da298fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:56.662412 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f054a049-5747-4edc-9e90-aac79da298fb None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:56.671919 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 164/326] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 22:34:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:56.677553 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecae9c6-72ff-48b6-8f6c-44b2db8bc0e2 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:56.723679 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:56.786657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecae9c6-72ff-48b6-8f6c-44b2db8bc0e2 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:56.822379 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255c588e-3985-4bc6-a4f4-a64852b00316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:56.823298 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c588e-3985-4bc6-a4f4-a64852b00316 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:56.823577 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c588e-3985-4bc6-a4f4-a64852b00316 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:56.823866 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c588e-3985-4bc6-a4f4-a64852b00316 None admin] PATH_INFO: `/v3/users/835e88eebc8c4731afef79c594c6413c` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:56.825056 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255c588e-3985-4bc6-a4f4-a64852b00316 None admin] RBAC: Authorizing `identity:get_user(user_id=835e88eebc8c4731afef79c594c6413c)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:56.826494 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255c588e-3985-4bc6-a4f4-a64852b00316 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:56.828452 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 163/327] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 22:34:56 2022] GET /identity/v3/users/835e88eebc8c4731afef79c594c6413c => generated 293 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:56.834212 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f054a049-5747-4edc-9e90-aac79da298fb None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:56.903110 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5d5fb-e90a-4d98-856c-8d855a7f2135 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:56.903412 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5d5fb-e90a-4d98-856c-8d855a7f2135 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:56.903672 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5d5fb-e90a-4d98-856c-8d855a7f2135 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:56.905197 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 164/328] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:34:56.910361 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e2be35-cfc0-40c7-aa3f-b1d35175ec1b None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:56.910647 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e2be35-cfc0-40c7-aa3f-b1d35175ec1b None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:56.911001 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e2be35-cfc0-40c7-aa3f-b1d35175ec1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:56.937814 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-65e2be35-cfc0-40c7-aa3f-b1d35175ec1b None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:56.940189 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-65e2be35-cfc0-40c7-aa3f-b1d35175ec1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:56.953171 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f054a049-5747-4edc-9e90-aac79da298fb None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:56.998677 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:57.007689 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1a7950-8f22-408c-896b-b81ac8627b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:57.009618 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1a7950-8f22-408c-896b-b81ac8627b30 None admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:57.021345 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1a7950-8f22-408c-896b-b81ac8627b30 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:57.021345 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1a7950-8f22-408c-896b-b81ac8627b30 None admin] PATH_INFO: `/v3/system/users/835e88eebc8c4731afef79c594c6413c/roles/6ea16f1a323a473e81c1b161134d7512` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:57.021345 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1a7950-8f22-408c-896b-b81ac8627b30 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=835e88eebc8c4731afef79c594c6413c, role_id=6ea16f1a323a473e81c1b161134d7512)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:57.021345 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1a7950-8f22-408c-896b-b81ac8627b30 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:57.039550 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 165/329] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:34:57.063671 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e2be35-cfc0-40c7-aa3f-b1d35175ec1b None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:57.092661 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 165/330] 10.222.0.52 () {58 vars in 1334 bytes} [Thu Jul 21 22:34:56 2022] PUT /identity/v3/system/users/835e88eebc8c4731afef79c594c6413c/roles/6ea16f1a323a473e81c1b161134d7512 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:34:57.119245 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:57.213517 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e2be35-cfc0-40c7-aa3f-b1d35175ec1b None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:57.255234 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec7fcef-44a6-43cc-88b7-3eaf037c95b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:57.256291 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec7fcef-44a6-43cc-88b7-3eaf037c95b8 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:57.256720 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec7fcef-44a6-43cc-88b7-3eaf037c95b8 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:57.257133 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec7fcef-44a6-43cc-88b7-3eaf037c95b8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:57.274177 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec7fcef-44a6-43cc-88b7-3eaf037c95b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:34:57.283327 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-4ec7fcef-44a6-43cc-88b7-3eaf037c95b8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:34:57.283327 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:34:57.283327 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:34:57.283327 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:34:57.283327 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:34:57.283327 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:34:57.283327 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:34:57.283327 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:34:57.283327 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:34:57.283327 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:34:57.283327 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:34:57.283327 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:34:57.283327 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:34:57.283327 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:34:57.283327 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:34:57.283327 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:34:57.284471 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:34:57.284471 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:34:57.284471 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:34:57.284471 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:34:57.284471 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:34:57.284471 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:34:57.284471 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:34:57.284471 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:34:57.284471 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:34:57.284471 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:34:57.284471 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:34:57.284471 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:34:57.284471 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:34:57.284471 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:34:57.284471 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:34:57.284471 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:34:57.285453 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:34:57.285453 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:34:57.285453 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:34:57.285453 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:34:57.285453 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:34:57.285453 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:34:57.285453 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:34:57.285453 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:34:57.285453 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:34:57.285453 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:34:57.285453 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:34:57.285453 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:34:57.287648 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 166/331] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:34:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:34:57.294275 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1a7950-8f22-408c-896b-b81ac8627b30 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:57.423509 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1a7950-8f22-408c-896b-b81ac8627b30 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:57.464971 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771a2224-b0bc-47bd-94c8-ca3defe16080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:57.465921 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a2224-b0bc-47bd-94c8-ca3defe16080 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:57.466346 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a2224-b0bc-47bd-94c8-ca3defe16080 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:57.466759 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a2224-b0bc-47bd-94c8-ca3defe16080 None admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:57.467679 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771a2224-b0bc-47bd-94c8-ca3defe16080 None admin] RBAC: Adding query filter params (name=member) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:57.468125 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771a2224-b0bc-47bd-94c8-ca3defe16080 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:57.469666 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771a2224-b0bc-47bd-94c8-ca3defe16080 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:57.480209 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 166/332] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 22:34:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:57.486044 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec7fcef-44a6-43cc-88b7-3eaf037c95b8 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:57.609545 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:57.613857 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec7fcef-44a6-43cc-88b7-3eaf037c95b8 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:57.655369 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd942ce-deee-4d26-b222-18abed9883bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:57.656245 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd942ce-deee-4d26-b222-18abed9883bb None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:57.656524 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd942ce-deee-4d26-b222-18abed9883bb None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:57.656794 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd942ce-deee-4d26-b222-18abed9883bb None admin] PATH_INFO: `/v3/users/9a7047999f054d4f89e4137b0a2f9994` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:57.657987 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd942ce-deee-4d26-b222-18abed9883bb None admin] RBAC: Authorizing `identity:get_user(user_id=9a7047999f054d4f89e4137b0a2f9994)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:57.659465 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd942ce-deee-4d26-b222-18abed9883bb None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:57.661096 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 167/333] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 22:34:57 2022] GET /identity/v3/users/9a7047999f054d4f89e4137b0a2f9994 => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:57.667648 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771a2224-b0bc-47bd-94c8-ca3defe16080 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:57.736999 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:57.786162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771a2224-b0bc-47bd-94c8-ca3defe16080 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:57.825794 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6dee0f-81d6-42bd-b388-6e1a2c348e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:57.826461 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6dee0f-81d6-42bd-b388-6e1a2c348e8f None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:57.826744 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6dee0f-81d6-42bd-b388-6e1a2c348e8f None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:57.827126 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6dee0f-81d6-42bd-b388-6e1a2c348e8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:57.827989 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6dee0f-81d6-42bd-b388-6e1a2c348e8f None admin] RBAC: Adding query filter params (role.id=53d3bdbe9cdf4d35bc73854a33df8e45, user.id=9a7047999f054d4f89e4137b0a2f9994, scope.system=all) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:57.828305 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6dee0f-81d6-42bd-b388-6e1a2c348e8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:57.829824 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6dee0f-81d6-42bd-b388-6e1a2c348e8f None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:57.847412 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 167/334] 10.222.0.52 () {56 vars in 1308 bytes} [Thu Jul 21 22:34:57 2022] GET /identity/v3/role_assignments?user.id=9a7047999f054d4f89e4137b0a2f9994&scope.system=all&role.id=53d3bdbe9cdf4d35bc73854a33df8e45 => generated 510 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:59.000254 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18791bce-c01c-4ef7-9b43-85537b398edb None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:59.000793 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18791bce-c01c-4ef7-9b43-85537b398edb None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:59.000793 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18791bce-c01c-4ef7-9b43-85537b398edb None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:59.002171 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 168/335] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:59.007680 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f44b78-cce5-4144-be2c-b9474bb6e324 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:59.008063 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f44b78-cce5-4144-be2c-b9474bb6e324 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:59.008254 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f44b78-cce5-4144-be2c-b9474bb6e324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:59.031333 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-59f44b78-cce5-4144-be2c-b9474bb6e324 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:59.032548 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-59f44b78-cce5-4144-be2c-b9474bb6e324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:59.126212 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 168/336] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:34:59.170322 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18791bce-c01c-4ef7-9b43-85537b398edb None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:59.291155 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18791bce-c01c-4ef7-9b43-85537b398edb None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:59.329645 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6979bb-b03d-4227-8cb0-5ade1684d63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:59.330463 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6979bb-b03d-4227-8cb0-5ade1684d63e None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:59.330812 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6979bb-b03d-4227-8cb0-5ade1684d63e None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:59.331099 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6979bb-b03d-4227-8cb0-5ade1684d63e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:59.347767 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6979bb-b03d-4227-8cb0-5ade1684d63e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:34:59.356413 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-dd6979bb-b03d-4227-8cb0-5ade1684d63e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:34:59.356413 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:34:59.356413 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:34:59.356413 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:34:59.356413 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:34:59.356413 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:34:59.356413 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:34:59.356413 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:34:59.356413 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:34:59.356413 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:34:59.356413 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:34:59.356413 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:34:59.356413 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:34:59.356413 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:34:59.356413 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:34:59.356413 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:34:59.357261 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:34:59.357261 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:34:59.357261 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:34:59.357261 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:34:59.357261 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:34:59.357261 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:34:59.357261 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:34:59.357261 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:34:59.357261 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:34:59.357261 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:34:59.357261 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:34:59.357261 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:34:59.357261 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:34:59.357261 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:34:59.357261 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:34:59.357261 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:34:59.358052 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:34:59.358052 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:34:59.358052 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:34:59.358052 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:34:59.358052 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:34:59.358052 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:34:59.358052 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:34:59.358052 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:34:59.358052 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:34:59.358052 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:34:59.358052 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:34:59.358052 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:34:59.358052 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 169/337] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 21 22:34:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:34:59.363576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f44b78-cce5-4144-be2c-b9474bb6e324 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:59.416750 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:59.482380 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f44b78-cce5-4144-be2c-b9474bb6e324 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:59.528145 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f95b56-f5eb-4735-b51f-b33014261394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:59.528836 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f95b56-f5eb-4735-b51f-b33014261394 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:59.529120 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f95b56-f5eb-4735-b51f-b33014261394 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:59.529399 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f95b56-f5eb-4735-b51f-b33014261394 None admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:59.530246 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f95b56-f5eb-4735-b51f-b33014261394 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:34:59.530556 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f95b56-f5eb-4735-b51f-b33014261394 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:59.532102 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f95b56-f5eb-4735-b51f-b33014261394 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:59.545176 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 169/338] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:34:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:34:59.550409 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6979bb-b03d-4227-8cb0-5ade1684d63e None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:59.561510 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecbd887-3085-4613-9715-dc2dc8d4145b None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:59.561795 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecbd887-3085-4613-9715-dc2dc8d4145b None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:59.562071 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecbd887-3085-4613-9715-dc2dc8d4145b None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:59.563584 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 170/339] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:34:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:34:59.569434 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a5414e-3623-418e-861a-14064111c5f8 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:59.569729 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a5414e-3623-418e-861a-14064111c5f8 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:59.570015 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a5414e-3623-418e-861a-14064111c5f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:59.592554 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-b4a5414e-3623-418e-861a-14064111c5f8 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:34:59.593753 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-b4a5414e-3623-418e-861a-14064111c5f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:34:59.599131 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:59.725834 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6979bb-b03d-4227-8cb0-5ade1684d63e None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:34:59.750399 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 171/340] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:34:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:34:59.786928 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffcfd28-b66d-48a7-8ead-fdb87fe44a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:34:59.787593 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffcfd28-b66d-48a7-8ead-fdb87fe44a88 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:34:59.787878 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffcfd28-b66d-48a7-8ead-fdb87fe44a88 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:34:59.788156 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffcfd28-b66d-48a7-8ead-fdb87fe44a88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:34:59.808194 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a5414e-3623-418e-861a-14064111c5f8 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:34:59.873196 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:34:59.909646 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffcfd28-b66d-48a7-8ead-fdb87fe44a88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:34:59.911140 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffcfd28-b66d-48a7-8ead-fdb87fe44a88 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:34:59.965169 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:00.005756 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a5414e-3623-418e-861a-14064111c5f8 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:00.068437 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b939aa-ca5c-4657-9097-59839df7dbd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:00.069160 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b939aa-ca5c-4657-9097-59839df7dbd9 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:00.069444 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b939aa-ca5c-4657-9097-59839df7dbd9 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:00.069723 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b939aa-ca5c-4657-9097-59839df7dbd9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:00.093532 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b939aa-ca5c-4657-9097-59839df7dbd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:35:00.106324 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-b0b939aa-ca5c-4657-9097-59839df7dbd9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:35:00.106324 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:00.106324 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:00.106324 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:00.106324 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:00.106324 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:00.106324 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:00.106324 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:00.106324 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:00.106324 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:00.106324 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:00.106324 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:00.106324 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:35:00.106324 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:35:00.106324 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:35:00.106324 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:35:00.107217 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:35:00.107217 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:35:00.107217 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:35:00.107217 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:35:00.107217 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:00.107217 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:00.107217 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:00.107217 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:00.107217 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:00.107217 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:00.107217 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:00.107217 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:00.107217 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:00.107217 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:00.107217 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:00.107217 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:00.107992 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:00.107992 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:00.107992 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:00.107992 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:00.107992 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:35:00.107992 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:35:00.107992 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:35:00.107992 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:35:00.107992 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:35:00.107992 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:35:00.107992 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:35:00.107992 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:00.107992 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 172/341] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:34:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:35:00.109114 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847eabbd-d096-4ff1-82ab-46a1b782cb07 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:00.109114 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847eabbd-d096-4ff1-82ab-46a1b782cb07 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:00.109355 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847eabbd-d096-4ff1-82ab-46a1b782cb07 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:00.110800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 173/342] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:00.115441 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 170/343] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:34:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:35:00.116166 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36837c4e-68ec-4715-956a-b0d72b474927 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:00.116455 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36837c4e-68ec-4715-956a-b0d72b474927 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:00.116738 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36837c4e-68ec-4715-956a-b0d72b474927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:00.121406 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffcfd28-b66d-48a7-8ead-fdb87fe44a88 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:00.137581 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-36837c4e-68ec-4715-956a-b0d72b474927 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:00.138964 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-36837c4e-68ec-4715-956a-b0d72b474927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:00.215980 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 174/344] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:35:00.216991 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36837c4e-68ec-4715-956a-b0d72b474927 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:00.258517 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:00.284997 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffcfd28-b66d-48a7-8ead-fdb87fe44a88 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:00.336510 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:00.355877 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36837c4e-68ec-4715-956a-b0d72b474927 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:00.364577 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc0fd07-9de8-4092-ae92-0b50fb9f296f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:00.368519 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc0fd07-9de8-4092-ae92-0b50fb9f296f None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:00.368828 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc0fd07-9de8-4092-ae92-0b50fb9f296f None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:00.369078 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc0fd07-9de8-4092-ae92-0b50fb9f296f None admin] PATH_INFO: `/v3/projects/6edc20faecff41bf9b7ecc5ac4bc774e` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:00.370484 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc0fd07-9de8-4092-ae92-0b50fb9f296f None admin] RBAC: Authorizing `identity:get_project(project_id=6edc20faecff41bf9b7ecc5ac4bc774e)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:00.372028 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc0fd07-9de8-4092-ae92-0b50fb9f296f None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:00.374186 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 171/345] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:35:00 2022] GET /identity/v3/projects/6edc20faecff41bf9b7ecc5ac4bc774e => generated 298 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:35:00.375099 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc0fd07-9de8-4092-ae92-0b50fb9f296f None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:00.397674 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7403607-12a5-4755-bb61-4716bbdb68e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:00.398692 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7403607-12a5-4755-bb61-4716bbdb68e2 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:00.399223 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7403607-12a5-4755-bb61-4716bbdb68e2 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:00.400251 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7403607-12a5-4755-bb61-4716bbdb68e2 None admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:00.401229 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7403607-12a5-4755-bb61-4716bbdb68e2 None admin] RBAC: Adding query filter params (name=member) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:00.401731 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7403607-12a5-4755-bb61-4716bbdb68e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:00.403356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7403607-12a5-4755-bb61-4716bbdb68e2 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:00.412292 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 175/346] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 22:35:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:00.413900 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb2769d-5696-41a0-b131-2619f833e56b None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:00.414386 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb2769d-5696-41a0-b131-2619f833e56b None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:00.414919 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb2769d-5696-41a0-b131-2619f833e56b None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:00.416389 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 176/347] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:00.417534 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb2769d-5696-41a0-b131-2619f833e56b None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:00.509742 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc0fd07-9de8-4092-ae92-0b50fb9f296f None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:00.536087 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:00.554257 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3fa618-bcac-4cd5-9651-bd40826a67cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:00.554257 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3fa618-bcac-4cd5-9651-bd40826a67cb None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:00.554257 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3fa618-bcac-4cd5-9651-bd40826a67cb None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:00.554257 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3fa618-bcac-4cd5-9651-bd40826a67cb None admin] PATH_INFO: `/v3/roles/member` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:00.554699 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb2769d-5696-41a0-b131-2619f833e56b None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:00.573467 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3fa618-bcac-4cd5-9651-bd40826a67cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:35:00.581385 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-da3fa618-bcac-4cd5-9651-bd40826a67cb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:35:00.581385 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:00.581385 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:00.581385 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:00.581385 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:00.581385 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:00.581385 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:00.581385 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:00.581385 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:00.581385 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:00.581385 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:00.581385 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:00.581385 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:35:00.581385 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:35:00.581385 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:35:00.581385 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:35:00.582544 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:35:00.582544 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:35:00.582544 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:35:00.582544 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:35:00.582544 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:00.582544 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:00.582544 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:00.582544 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:00.582544 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:00.582544 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:00.582544 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:00.582544 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:00.582544 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:00.582544 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:00.582544 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:00.582544 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:00.583401 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:00.583401 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:00.583401 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:00.583401 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:00.583401 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:35:00.583401 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:35:00.583401 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:35:00.583401 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:35:00.583401 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:35:00.583401 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:35:00.583401 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:35:00.583401 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:00.583401 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 172/348] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:35:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:00.584527 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3fa618-bcac-4cd5-9651-bd40826a67cb None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:00.659589 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc92e0f3-24bb-4d37-afbe-b59b27240392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:00.660741 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92e0f3-24bb-4d37-afbe-b59b27240392 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:00.661202 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92e0f3-24bb-4d37-afbe-b59b27240392 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:00.661683 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92e0f3-24bb-4d37-afbe-b59b27240392 None admin] PATH_INFO: `/v3/groups/5f723cf655a24fbfbf12b2dc2aa3e0c7` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:00.672834 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc92e0f3-24bb-4d37-afbe-b59b27240392 None admin] RBAC: Authorizing `identity:get_group(group_id=5f723cf655a24fbfbf12b2dc2aa3e0c7)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:00.674567 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc92e0f3-24bb-4d37-afbe-b59b27240392 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:00.676283 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:00.676738 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 177/349] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jul 21 22:35:00 2022] GET /identity/v3/groups/5f723cf655a24fbfbf12b2dc2aa3e0c7 => generated 226 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:00.678714 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c219e-d397-4345-9a76-30de8b4a6fd7 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:00.679459 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c219e-d397-4345-9a76-30de8b4a6fd7 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:00.680159 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c219e-d397-4345-9a76-30de8b4a6fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:00.703026 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-799c219e-d397-4345-9a76-30de8b4a6fd7 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:00.704595 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-799c219e-d397-4345-9a76-30de8b4a6fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:00.724346 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3fa618-bcac-4cd5-9651-bd40826a67cb None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:00.761635 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c39f2-eff8-4cf1-9bb3-438f3373abf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:00.761635 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c39f2-eff8-4cf1-9bb3-438f3373abf9 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:00.761635 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c39f2-eff8-4cf1-9bb3-438f3373abf9 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:00.761983 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c39f2-eff8-4cf1-9bb3-438f3373abf9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:00.783894 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 178/350] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:35:00.785059 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799c219e-d397-4345-9a76-30de8b4a6fd7 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:00.836491 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4c39f2-eff8-4cf1-9bb3-438f3373abf9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:00.838526 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4c39f2-eff8-4cf1-9bb3-438f3373abf9 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:00.873994 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:00.895529 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799c219e-d397-4345-9a76-30de8b4a6fd7 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:00.909420 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:00.937364 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2fde48-cec4-4527-b422-ad4b84c2cf92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:00.938469 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2fde48-cec4-4527-b422-ad4b84c2cf92 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:00.939011 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2fde48-cec4-4527-b422-ad4b84c2cf92 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:00.939491 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2fde48-cec4-4527-b422-ad4b84c2cf92 None admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:00.940448 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2fde48-cec4-4527-b422-ad4b84c2cf92 None admin] RBAC: Adding query filter params (name=member) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:00.940977 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2fde48-cec4-4527-b422-ad4b84c2cf92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:00.942556 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2fde48-cec4-4527-b422-ad4b84c2cf92 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:00.950649 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 173/351] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:35:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:35:00.951687 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c39f2-eff8-4cf1-9bb3-438f3373abf9 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:00.953417 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 179/352] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 22:35:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:00.954437 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2fde48-cec4-4527-b422-ad4b84c2cf92 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:01.072719 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2fde48-cec4-4527-b422-ad4b84c2cf92 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:01.080141 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c39f2-eff8-4cf1-9bb3-438f3373abf9 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:01.123118 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eab9b3-150c-41a8-9585-422c5b056491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:01.123926 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eab9b3-150c-41a8-9585-422c5b056491 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:01.124213 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eab9b3-150c-41a8-9585-422c5b056491 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:01.124491 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eab9b3-150c-41a8-9585-422c5b056491 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:01.125108 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89d9d54-5938-4628-afba-54bb10153555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:01.125775 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d9d54-5938-4628-afba-54bb10153555 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:01.126056 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d9d54-5938-4628-afba-54bb10153555 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:01.126504 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d9d54-5938-4628-afba-54bb10153555 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:01.127695 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89d9d54-5938-4628-afba-54bb10153555 None admin] RBAC: Adding query filter params (group.id=5f723cf655a24fbfbf12b2dc2aa3e0c7, role.id=b1849c34fc37410098b94ff687ab0055, scope.project.id=6edc20faecff41bf9b7ecc5ac4bc774e) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:01.128014 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89d9d54-5938-4628-afba-54bb10153555 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:01.129622 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89d9d54-5938-4628-afba-54bb10153555 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:01.136846 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 174/353] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Jul 21 22:35:00 2022] GET /identity/v3/role_assignments?group.id=5f723cf655a24fbfbf12b2dc2aa3e0c7&scope.project.id=6edc20faecff41bf9b7ecc5ac4bc774e&role.id=b1849c34fc37410098b94ff687ab0055 => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:01.137254 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eab9b3-150c-41a8-9585-422c5b056491 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:35:01.137652 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89d9d54-5938-4628-afba-54bb10153555 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:01.145120 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-d8eab9b3-150c-41a8-9585-422c5b056491 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:35:01.145120 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:01.145120 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:01.145120 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:01.145120 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:01.145120 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:01.145120 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:01.145120 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:01.145120 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:01.145120 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:01.145120 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:01.145120 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:01.145120 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:35:01.145120 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:35:01.145120 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:35:01.145120 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:35:01.146079 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:35:01.146079 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:35:01.146079 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:35:01.146079 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:35:01.146079 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:01.146079 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:01.146079 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:01.146079 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:01.146079 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:01.146079 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:01.146079 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:01.146079 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:01.146079 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:01.146079 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:01.146079 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:01.146079 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:01.146896 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:01.146896 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:01.146896 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:01.146896 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:01.146896 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:35:01.146896 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:35:01.146896 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:35:01.146896 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:35:01.146896 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:35:01.146896 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:35:01.146896 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 22:35:01.146896 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:01.146896 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 180/354] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:35:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:01.148167 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eab9b3-150c-41a8-9585-422c5b056491 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:01.166155 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:01.168957 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:01.264856 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89d9d54-5938-4628-afba-54bb10153555 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:01.273626 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eab9b3-150c-41a8-9585-422c5b056491 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:01.325180 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095f2a89-7f30-476b-85e1-e5dd06dba2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:01.325444 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f2a89-7f30-476b-85e1-e5dd06dba2ae None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:01.325875 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f2a89-7f30-476b-85e1-e5dd06dba2ae None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:01.326375 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f2a89-7f30-476b-85e1-e5dd06dba2ae None admin] PATH_INFO: `/v3/projects/a54cbea0067945899d2c8c75b7973660` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:01.328004 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095f2a89-7f30-476b-85e1-e5dd06dba2ae None admin] RBAC: Authorizing `identity:get_project(project_id=a54cbea0067945899d2c8c75b7973660)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:01.329452 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095f2a89-7f30-476b-85e1-e5dd06dba2ae None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:01.330870 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b48e0c-4496-4568-875c-b3d0975a751d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:01.331363 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 175/355] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:35:01 2022] GET /identity/v3/projects/a54cbea0067945899d2c8c75b7973660 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:01.331982 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b48e0c-4496-4568-875c-b3d0975a751d None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:01.332271 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b48e0c-4496-4568-875c-b3d0975a751d None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:01.332551 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b48e0c-4496-4568-875c-b3d0975a751d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:01.332654 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a0df9-1d62-46b3-a884-de2c5576441b None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:01.332843 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a0df9-1d62-46b3-a884-de2c5576441b None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:01.333155 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a0df9-1d62-46b3-a884-de2c5576441b None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:01.334395 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 176/356] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:01.335121 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88a0df9-1d62-46b3-a884-de2c5576441b None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:01.431274 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b48e0c-4496-4568-875c-b3d0975a751d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:01.432753 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b48e0c-4496-4568-875c-b3d0975a751d None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:01.436995 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:01.443988 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:01.462337 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88a0df9-1d62-46b3-a884-de2c5576441b None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:01.516615 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b4f57f-3a12-4f6a-b8e9-0cd933a09748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:01.516797 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4f57f-3a12-4f6a-b8e9-0cd933a09748 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:01.517020 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4f57f-3a12-4f6a-b8e9-0cd933a09748 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:01.517373 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4f57f-3a12-4f6a-b8e9-0cd933a09748 None admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:01.518292 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b4f57f-3a12-4f6a-b8e9-0cd933a09748 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:01.518292 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b4f57f-3a12-4f6a-b8e9-0cd933a09748 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:01.519689 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b4f57f-3a12-4f6a-b8e9-0cd933a09748 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:01.533162 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 177/357] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 22:35:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:01.534018 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b4f57f-3a12-4f6a-b8e9-0cd933a09748 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:01.589414 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 181/358] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:35:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:35:01.589414 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d3aa7-b285-4d8c-a054-7404eddc8486 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:01.589414 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d3aa7-b285-4d8c-a054-7404eddc8486 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:01.589414 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d3aa7-b285-4d8c-a054-7404eddc8486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:01.609494 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-804d3aa7-b285-4d8c-a054-7404eddc8486 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:01.610705 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-804d3aa7-b285-4d8c-a054-7404eddc8486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:01.666624 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b4f57f-3a12-4f6a-b8e9-0cd933a09748 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:01.706510 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 182/359] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:35:01.707968 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4538eb-3e77-436d-bd7e-86591c0e50d2 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:01.708261 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4538eb-3e77-436d-bd7e-86591c0e50d2 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:01.708545 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4538eb-3e77-436d-bd7e-86591c0e50d2 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:01.708798 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc1e3a7-001a-40fe-940b-0a2855370590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:01.709839 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc1e3a7-001a-40fe-940b-0a2855370590 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:01.710051 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 183/360] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:01.710412 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc1e3a7-001a-40fe-940b-0a2855370590 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:01.710752 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4538eb-3e77-436d-bd7e-86591c0e50d2 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:01.711012 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc1e3a7-001a-40fe-940b-0a2855370590 None admin] PATH_INFO: `/v3/groups/5f723cf655a24fbfbf12b2dc2aa3e0c7` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:01.716198 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc1e3a7-001a-40fe-940b-0a2855370590 None admin] RBAC: Authorizing `identity:get_group(group_id=5f723cf655a24fbfbf12b2dc2aa3e0c7)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:01.716980 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc1e3a7-001a-40fe-940b-0a2855370590 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:01.718744 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 178/361] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jul 21 22:35:01 2022] GET /identity/v3/groups/5f723cf655a24fbfbf12b2dc2aa3e0c7 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:01.719753 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc1e3a7-001a-40fe-940b-0a2855370590 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:01.747422 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:01.761443 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:01.840058 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4538eb-3e77-436d-bd7e-86591c0e50d2 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:01.857327 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc1e3a7-001a-40fe-940b-0a2855370590 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:01.882697 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b9aee9-b6ee-446f-b79f-d7465d8a466f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:01.883718 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b9aee9-b6ee-446f-b79f-d7465d8a466f None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:01.884065 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b9aee9-b6ee-446f-b79f-d7465d8a466f None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:01.884399 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b9aee9-b6ee-446f-b79f-d7465d8a466f None admin] PATH_INFO: `/v3/users/835e88eebc8c4731afef79c594c6413c` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:01.886119 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b9aee9-b6ee-446f-b79f-d7465d8a466f None admin] RBAC: Authorizing `identity:get_user(user_id=835e88eebc8c4731afef79c594c6413c)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:01.887945 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b9aee9-b6ee-446f-b79f-d7465d8a466f None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:01.890279 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 184/362] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 22:35:01 2022] GET /identity/v3/users/835e88eebc8c4731afef79c594c6413c => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:01.891359 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b767f9-40fa-4f4d-9d1b-b1d282b9260d None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:01.891719 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b767f9-40fa-4f4d-9d1b-b1d282b9260d None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:01.892051 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b767f9-40fa-4f4d-9d1b-b1d282b9260d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:01.907721 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd4c56-2c5e-41f3-951c-881f890b1252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:01.908630 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd4c56-2c5e-41f3-951c-881f890b1252 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:01.908919 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd4c56-2c5e-41f3-951c-881f890b1252 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:01.909194 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd4c56-2c5e-41f3-951c-881f890b1252 None admin] PATH_INFO: `/v3/projects/6edc20faecff41bf9b7ecc5ac4bc774e` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:01.910488 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffd4c56-2c5e-41f3-951c-881f890b1252 None admin] RBAC: Authorizing `identity:get_project(project_id=6edc20faecff41bf9b7ecc5ac4bc774e)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:01.916045 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-00b767f9-40fa-4f4d-9d1b-b1d282b9260d None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:01.916192 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffd4c56-2c5e-41f3-951c-881f890b1252 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:01.916289 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-00b767f9-40fa-4f4d-9d1b-b1d282b9260d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:01.916382 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 179/363] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:35:01 2022] GET /identity/v3/projects/6edc20faecff41bf9b7ecc5ac4bc774e => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:01.916382 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd4c56-2c5e-41f3-951c-881f890b1252 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:02.007590 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 185/364] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:35:02.009057 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b767f9-40fa-4f4d-9d1b-b1d282b9260d None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:02.027111 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:02.043671 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:02.046843 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd4c56-2c5e-41f3-951c-881f890b1252 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:02.090038 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524957b8-8646-4f0b-8abd-a0e7d976a154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:02.091036 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524957b8-8646-4f0b-8abd-a0e7d976a154 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:02.091335 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524957b8-8646-4f0b-8abd-a0e7d976a154 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:02.091693 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524957b8-8646-4f0b-8abd-a0e7d976a154 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:02.092607 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524957b8-8646-4f0b-8abd-a0e7d976a154 None admin] RBAC: Adding query filter params (group.id=5f723cf655a24fbfbf12b2dc2aa3e0c7, role.id=53d3bdbe9cdf4d35bc73854a33df8e45, scope.project.id=a54cbea0067945899d2c8c75b7973660) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:02.095484 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524957b8-8646-4f0b-8abd-a0e7d976a154 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:02.095484 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524957b8-8646-4f0b-8abd-a0e7d976a154 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:02.101543 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 180/365] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Jul 21 22:35:01 2022] GET /identity/v3/role_assignments?group.id=5f723cf655a24fbfbf12b2dc2aa3e0c7&scope.project.id=a54cbea0067945899d2c8c75b7973660&role.id=53d3bdbe9cdf4d35bc73854a33df8e45 => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:02.102184 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524957b8-8646-4f0b-8abd-a0e7d976a154 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:02.157278 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b767f9-40fa-4f4d-9d1b-b1d282b9260d None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:02.212165 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104db13d-a57b-4723-9642-dc919e6f8b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:02.213917 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104db13d-a57b-4723-9642-dc919e6f8b46 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:02.214659 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104db13d-a57b-4723-9642-dc919e6f8b46 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:02.215175 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104db13d-a57b-4723-9642-dc919e6f8b46 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:02.232543 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104db13d-a57b-4723-9642-dc919e6f8b46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:35:02.245063 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-104db13d-a57b-4723-9642-dc919e6f8b46 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:35:02.245063 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:02.245063 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:02.245063 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:02.245063 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:02.245063 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:02.245063 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:02.245063 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:02.245063 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:02.245063 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:02.245063 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:02.245063 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:02.245063 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:35:02.245063 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:35:02.245063 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:35:02.245063 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:35:02.246045 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:35:02.246045 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:35:02.246045 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:35:02.246045 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:35:02.246045 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:02.246045 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:02.246045 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:02.246045 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:02.246045 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:02.246045 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:02.246045 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:02.246045 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:02.246045 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:02.246045 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:02.246045 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:02.246045 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:02.246882 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:02.246882 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:02.246882 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:02.246882 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:02.246882 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:35:02.246882 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:35:02.246882 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:35:02.246882 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:35:02.246882 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:35:02.246882 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:35:02.246882 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:35:02.246882 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:02.248218 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 186/366] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 21 22:35:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:02.249033 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104db13d-a57b-4723-9642-dc919e6f8b46 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:02.276897 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524957b8-8646-4f0b-8abd-a0e7d976a154 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:02.338923 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949f5f69-888b-4cc7-ac35-f5a96060d222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:02.340063 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f5f69-888b-4cc7-ac35-f5a96060d222 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:02.340552 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f5f69-888b-4cc7-ac35-f5a96060d222 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:02.341102 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f5f69-888b-4cc7-ac35-f5a96060d222 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:02.342022 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949f5f69-888b-4cc7-ac35-f5a96060d222 None admin] RBAC: Adding query filter params (role.id=6ea16f1a323a473e81c1b161134d7512, user.id=835e88eebc8c4731afef79c594c6413c, scope.system=all) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:02.342440 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949f5f69-888b-4cc7-ac35-f5a96060d222 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:02.344006 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949f5f69-888b-4cc7-ac35-f5a96060d222 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:02.362042 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 181/367] 10.222.0.52 () {56 vars in 1308 bytes} [Thu Jul 21 22:35:02 2022] GET /identity/v3/role_assignments?user.id=835e88eebc8c4731afef79c594c6413c&scope.system=all&role.id=6ea16f1a323a473e81c1b161134d7512 => generated 510 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:02.362042 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949f5f69-888b-4cc7-ac35-f5a96060d222 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:02.371152 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:02.441510 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:02.452383 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104db13d-a57b-4723-9642-dc919e6f8b46 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:02.498465 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456c2fa1-44b1-4e56-9275-10cf5f6a26ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:02.499377 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c2fa1-44b1-4e56-9275-10cf5f6a26ce None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:02.499664 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c2fa1-44b1-4e56-9275-10cf5f6a26ce None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:02.499944 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c2fa1-44b1-4e56-9275-10cf5f6a26ce None admin] PATH_INFO: `/v3/groups/5f723cf655a24fbfbf12b2dc2aa3e0c7` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:02.501154 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456c2fa1-44b1-4e56-9275-10cf5f6a26ce None admin] RBAC: Authorizing `identity:get_group(group_id=5f723cf655a24fbfbf12b2dc2aa3e0c7)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:02.502669 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456c2fa1-44b1-4e56-9275-10cf5f6a26ce None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:02.504368 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 187/368] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jul 21 22:35:02 2022] GET /identity/v3/groups/5f723cf655a24fbfbf12b2dc2aa3e0c7 => generated 226 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:02.505179 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456c2fa1-44b1-4e56-9275-10cf5f6a26ce None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:02.560996 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949f5f69-888b-4cc7-ac35-f5a96060d222 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:02.620636 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456c2fa1-44b1-4e56-9275-10cf5f6a26ce None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:02.644881 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e361dedf-0c6e-420e-bbb0-79b80207a3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:02.646191 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361dedf-0c6e-420e-bbb0-79b80207a3a8 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:02.646711 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361dedf-0c6e-420e-bbb0-79b80207a3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:02.647391 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361dedf-0c6e-420e-bbb0-79b80207a3a8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:02.660789 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa2e5b-9e03-47bd-aed9-cf9cb4bb7420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:02.662297 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa2e5b-9e03-47bd-aed9-cf9cb4bb7420 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:02.662872 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa2e5b-9e03-47bd-aed9-cf9cb4bb7420 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:02.663420 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa2e5b-9e03-47bd-aed9-cf9cb4bb7420 None admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:02.664361 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e361dedf-0c6e-420e-bbb0-79b80207a3a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:35:02.666138 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fa2e5b-9e03-47bd-aed9-cf9cb4bb7420 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:02.666822 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fa2e5b-9e03-47bd-aed9-cf9cb4bb7420 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:02.668546 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fa2e5b-9e03-47bd-aed9-cf9cb4bb7420 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:02.673544 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-e361dedf-0c6e-420e-bbb0-79b80207a3a8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:35:02.673544 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:02.673544 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:02.673544 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:02.673544 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:02.673544 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:02.673544 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:02.673544 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:02.673544 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:02.673544 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:02.673544 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:02.673544 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:02.673544 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:35:02.673544 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:35:02.673544 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:35:02.673544 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:35:02.674522 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:35:02.674522 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:35:02.674522 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:35:02.674522 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:35:02.674522 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:02.674522 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:02.674522 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:02.674522 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:02.674522 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:02.674522 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:02.674522 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:02.674522 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:02.674522 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:02.674522 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:02.674522 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:02.674522 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:02.675374 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:02.675374 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:02.675374 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:02.675374 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:02.675374 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:35:02.675374 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:35:02.675374 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:35:02.675374 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:35:02.675374 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:35:02.675374 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:35:02.675374 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:35:02.675374 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:02.676044 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 182/369] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 22:35:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:02.676599 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e361dedf-0c6e-420e-bbb0-79b80207a3a8 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:02.689880 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 188/370] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:35:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:02.692331 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa2e5b-9e03-47bd-aed9-cf9cb4bb7420 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:02.731192 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:02.763438 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:02.814517 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e361dedf-0c6e-420e-bbb0-79b80207a3a8 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:02.840011 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa2e5b-9e03-47bd-aed9-cf9cb4bb7420 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:02.861358 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ef4134-a3dd-4eec-99bd-4ec0d38178e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:02.862329 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ef4134-a3dd-4eec-99bd-4ec0d38178e9 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:02.862752 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ef4134-a3dd-4eec-99bd-4ec0d38178e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:02.863262 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ef4134-a3dd-4eec-99bd-4ec0d38178e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:02.864671 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ef4134-a3dd-4eec-99bd-4ec0d38178e9 None admin] RBAC: Adding query filter params (group.id=5f723cf655a24fbfbf12b2dc2aa3e0c7, role.id=53d3bdbe9cdf4d35bc73854a33df8e45, scope.project.id=6edc20faecff41bf9b7ecc5ac4bc774e) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:02.865195 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ef4134-a3dd-4eec-99bd-4ec0d38178e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:02.866823 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ef4134-a3dd-4eec-99bd-4ec0d38178e9 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:02.876618 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 183/371] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Jul 21 22:35:02 2022] GET /identity/v3/role_assignments?group.id=5f723cf655a24fbfbf12b2dc2aa3e0c7&scope.project.id=6edc20faecff41bf9b7ecc5ac4bc774e&role.id=53d3bdbe9cdf4d35bc73854a33df8e45 => generated 260 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:02.877919 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ef4134-a3dd-4eec-99bd-4ec0d38178e9 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:02.887317 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e92da5-17a9-4bba-a016-c3a108eb4d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:02.888164 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e92da5-17a9-4bba-a016-c3a108eb4d57 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:02.888553 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e92da5-17a9-4bba-a016-c3a108eb4d57 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:02.888930 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e92da5-17a9-4bba-a016-c3a108eb4d57 None admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:02.889951 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e92da5-17a9-4bba-a016-c3a108eb4d57 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:02.890393 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e92da5-17a9-4bba-a016-c3a108eb4d57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:02.892105 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e92da5-17a9-4bba-a016-c3a108eb4d57 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:02.903207 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 189/372] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:35:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:02.914782 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e92da5-17a9-4bba-a016-c3a108eb4d57 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:03.013311 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ef4134-a3dd-4eec-99bd-4ec0d38178e9 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:03.029260 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:03.046488 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:03.047345 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e92da5-17a9-4bba-a016-c3a108eb4d57 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:03.053044 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443feb5a-ea8d-4549-9735-93c22143f591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:03.053719 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443feb5a-ea8d-4549-9735-93c22143f591 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:03.054007 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443feb5a-ea8d-4549-9735-93c22143f591 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:03.054286 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443feb5a-ea8d-4549-9735-93c22143f591 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:03.085259 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8513f8a3-c84c-4485-8950-96fb9c700257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:03.086158 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8513f8a3-c84c-4485-8950-96fb9c700257 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:03.087211 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8513f8a3-c84c-4485-8950-96fb9c700257 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:03.087644 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8513f8a3-c84c-4485-8950-96fb9c700257 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:03.131490 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443feb5a-ea8d-4549-9735-93c22143f591 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:03.132880 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443feb5a-ea8d-4549-9735-93c22143f591 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:03.167444 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8513f8a3-c84c-4485-8950-96fb9c700257 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:03.169523 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8513f8a3-c84c-4485-8950-96fb9c700257 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:03.251948 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 184/373] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:35:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:35:03.259620 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443feb5a-ea8d-4549-9735-93c22143f591 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:03.272039 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:03.294013 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 190/374] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:35:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 22:35:03.300228 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8513f8a3-c84c-4485-8950-96fb9c700257 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:03.309787 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:03.433495 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443feb5a-ea8d-4549-9735-93c22143f591 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:03.444220 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8513f8a3-c84c-4485-8950-96fb9c700257 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:03.490356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f06271-7f49-4c0d-95af-de8b78c96f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:03.494313 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f06271-7f49-4c0d-95af-de8b78c96f53 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:03.494313 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f06271-7f49-4c0d-95af-de8b78c96f53 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:03.494313 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f06271-7f49-4c0d-95af-de8b78c96f53 None admin] PATH_INFO: `/v3/projects/d47c52ce443d40eaa4e635852fb964cd` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:03.496159 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f06271-7f49-4c0d-95af-de8b78c96f53 None admin] RBAC: Authorizing `identity:get_project(project_id=d47c52ce443d40eaa4e635852fb964cd)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:03.498356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f06271-7f49-4c0d-95af-de8b78c96f53 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:03.502765 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 191/375] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:35:03 2022] GET /identity/v3/projects/d47c52ce443d40eaa4e635852fb964cd => generated 344 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:03.508192 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f06271-7f49-4c0d-95af-de8b78c96f53 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:03.512510 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d25059-ccc0-4212-950b-9430ac399085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:03.513383 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d25059-ccc0-4212-950b-9430ac399085 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:03.513780 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d25059-ccc0-4212-950b-9430ac399085 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:03.513974 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d25059-ccc0-4212-950b-9430ac399085 None admin] PATH_INFO: `/v3/projects/a54cbea0067945899d2c8c75b7973660` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:03.515849 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d25059-ccc0-4212-950b-9430ac399085 None admin] RBAC: Authorizing `identity:get_project(project_id=a54cbea0067945899d2c8c75b7973660)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:03.516593 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d25059-ccc0-4212-950b-9430ac399085 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:03.518636 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 185/376] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:35:03 2022] GET /identity/v3/projects/a54cbea0067945899d2c8c75b7973660 => generated 302 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:03.524397 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a232e0f-0c67-4444-b347-52c7a5747198 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:03.524681 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a232e0f-0c67-4444-b347-52c7a5747198 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:03.524961 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a232e0f-0c67-4444-b347-52c7a5747198 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:03.526290 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 186/377] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:03.527989 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a232e0f-0c67-4444-b347-52c7a5747198 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:03.582264 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:03.647361 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f06271-7f49-4c0d-95af-de8b78c96f53 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:03.652385 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:03.658115 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a232e0f-0c67-4444-b347-52c7a5747198 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:03.699578 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040ef86c-ab5d-4adf-af62-5895d4b9ad1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:03.700390 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ef86c-ab5d-4adf-af62-5895d4b9ad1b None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:03.700677 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ef86c-ab5d-4adf-af62-5895d4b9ad1b None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:03.700957 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ef86c-ab5d-4adf-af62-5895d4b9ad1b None admin] PATH_INFO: `/v3/groups/d77a5b66867941989c313e3a336978b3` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:03.707984 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02dac4d-3058-4451-a3a5-44a125c14a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:03.708881 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02dac4d-3058-4451-a3a5-44a125c14a35 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:03.709167 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02dac4d-3058-4451-a3a5-44a125c14a35 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:03.709446 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02dac4d-3058-4451-a3a5-44a125c14a35 None admin] PATH_INFO: `/v3/groups/5f723cf655a24fbfbf12b2dc2aa3e0c7` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:03.710974 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02dac4d-3058-4451-a3a5-44a125c14a35 None admin] RBAC: Authorizing `identity:get_group(group_id=5f723cf655a24fbfbf12b2dc2aa3e0c7)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:03.712983 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02dac4d-3058-4451-a3a5-44a125c14a35 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:03.714954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 187/378] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jul 21 22:35:03 2022] GET /identity/v3/groups/5f723cf655a24fbfbf12b2dc2aa3e0c7 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:03.718599 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040ef86c-ab5d-4adf-af62-5895d4b9ad1b None admin] RBAC: Authorizing `identity:get_group(group_id=d77a5b66867941989c313e3a336978b3)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:03.720124 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040ef86c-ab5d-4adf-af62-5895d4b9ad1b None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:03.720512 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4c8ed-6f3c-480d-982e-7679e5cfd21a None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:03.721111 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4c8ed-6f3c-480d-982e-7679e5cfd21a None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:03.721598 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4c8ed-6f3c-480d-982e-7679e5cfd21a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:03.722148 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 192/379] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jul 21 22:35:03 2022] GET /identity/v3/groups/d77a5b66867941989c313e3a336978b3 => generated 229 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:03.723044 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040ef86c-ab5d-4adf-af62-5895d4b9ad1b None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:03.747405 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-49a4c8ed-6f3c-480d-982e-7679e5cfd21a None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:03.748605 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-49a4c8ed-6f3c-480d-982e-7679e5cfd21a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:03.838806 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 188/380] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:35:03.838806 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a4c8ed-6f3c-480d-982e-7679e5cfd21a None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:03.855193 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040ef86c-ab5d-4adf-af62-5895d4b9ad1b None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:03.880948 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:03.907435 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad6cf2-7247-48c8-ad8d-b394bfa9e85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:03.909084 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad6cf2-7247-48c8-ad8d-b394bfa9e85d None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:03.909369 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad6cf2-7247-48c8-ad8d-b394bfa9e85d None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:03.909734 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad6cf2-7247-48c8-ad8d-b394bfa9e85d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:03.910846 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ad6cf2-7247-48c8-ad8d-b394bfa9e85d None admin] RBAC: Adding query filter params (group.id=5f723cf655a24fbfbf12b2dc2aa3e0c7, role.id=b1849c34fc37410098b94ff687ab0055, scope.project.id=a54cbea0067945899d2c8c75b7973660) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:03.911165 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ad6cf2-7247-48c8-ad8d-b394bfa9e85d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:03.912597 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ad6cf2-7247-48c8-ad8d-b394bfa9e85d None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:03.919854 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 193/381] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Jul 21 22:35:03 2022] GET /identity/v3/role_assignments?group.id=5f723cf655a24fbfbf12b2dc2aa3e0c7&scope.project.id=a54cbea0067945899d2c8c75b7973660&role.id=b1849c34fc37410098b94ff687ab0055 => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:03.920503 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad6cf2-7247-48c8-ad8d-b394bfa9e85d None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:03.980486 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:03.987445 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a4c8ed-6f3c-480d-982e-7679e5cfd21a None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:04.047783 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad6cf2-7247-48c8-ad8d-b394bfa9e85d None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:04.086167 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c952d02c-d1ca-4372-99db-61dd70b30f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:04.087054 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952d02c-d1ca-4372-99db-61dd70b30f20 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:04.087343 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952d02c-d1ca-4372-99db-61dd70b30f20 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:04.087622 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952d02c-d1ca-4372-99db-61dd70b30f20 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:04.096452 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015ae9d0-4b78-4fcd-ac7f-ae46cbe389c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:04.097144 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ae9d0-4b78-4fcd-ac7f-ae46cbe389c5 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:04.097431 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ae9d0-4b78-4fcd-ac7f-ae46cbe389c5 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:04.097711 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ae9d0-4b78-4fcd-ac7f-ae46cbe389c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:04.098568 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015ae9d0-4b78-4fcd-ac7f-ae46cbe389c5 None admin] RBAC: Adding query filter params (group.id=d77a5b66867941989c313e3a336978b3, role.id=d1ddee590a964ff1bbb4ee8716173021, scope.project.id=d47c52ce443d40eaa4e635852fb964cd) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:04.098919 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015ae9d0-4b78-4fcd-ac7f-ae46cbe389c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:04.099749 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c952d02c-d1ca-4372-99db-61dd70b30f20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:35:04.100393 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015ae9d0-4b78-4fcd-ac7f-ae46cbe389c5 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:04.109052 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-c952d02c-d1ca-4372-99db-61dd70b30f20 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:35:04.109052 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:04.109052 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:04.109052 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:04.109052 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:04.109052 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:04.109052 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:04.109052 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:04.109052 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:04.109052 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:04.109052 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:04.109052 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:04.109052 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:35:04.109052 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:35:04.109052 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:35:04.109052 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:35:04.109988 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:35:04.109988 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:35:04.109988 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:35:04.109988 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:35:04.109988 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:04.109988 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:04.109988 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:04.109988 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:04.109988 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:04.109988 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:04.109988 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:04.109988 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:04.109988 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:04.109988 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:04.109988 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:04.109988 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:04.110841 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:04.110841 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:04.110841 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:04.110841 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:04.110841 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:35:04.110841 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:35:04.110841 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:35:04.110841 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:35:04.110841 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:35:04.110841 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:35:04.110841 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:35:04.110841 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:04.110841 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 194/382] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 21 22:35:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:04.117107 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 189/383] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Jul 21 22:35:03 2022] GET /identity/v3/role_assignments?group.id=d77a5b66867941989c313e3a336978b3&scope.project.id=d47c52ce443d40eaa4e635852fb964cd&role.id=d1ddee590a964ff1bbb4ee8716173021 => generated 260 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:04.126891 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c952d02c-d1ca-4372-99db-61dd70b30f20 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:04.193720 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:04.280639 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c952d02c-d1ca-4372-99db-61dd70b30f20 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:04.326235 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57803bf9-e67e-47b7-b5da-ae8e6f5efe3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:04.326964 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57803bf9-e67e-47b7-b5da-ae8e6f5efe3d None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:04.327248 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57803bf9-e67e-47b7-b5da-ae8e6f5efe3d None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:04.327527 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57803bf9-e67e-47b7-b5da-ae8e6f5efe3d None admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:04.328307 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57803bf9-e67e-47b7-b5da-ae8e6f5efe3d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:04.328617 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57803bf9-e67e-47b7-b5da-ae8e6f5efe3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:04.330001 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57803bf9-e67e-47b7-b5da-ae8e6f5efe3d None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:04.340021 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 195/384] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:35:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:04.346576 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015ae9d0-4b78-4fcd-ac7f-ae46cbe389c5 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:04.485348 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015ae9d0-4b78-4fcd-ac7f-ae46cbe389c5 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:04.523622 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e00accf-1be4-43e0-8085-8a070662b0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:04.525192 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e00accf-1be4-43e0-8085-8a070662b0dc None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:04.525475 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e00accf-1be4-43e0-8085-8a070662b0dc None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:04.525748 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e00accf-1be4-43e0-8085-8a070662b0dc None admin] PATH_INFO: `/v3/groups/5f723cf655a24fbfbf12b2dc2aa3e0c7` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:04.527014 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e00accf-1be4-43e0-8085-8a070662b0dc None admin] RBAC: Authorizing `identity:get_group(group_id=5f723cf655a24fbfbf12b2dc2aa3e0c7)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:04.528507 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e00accf-1be4-43e0-8085-8a070662b0dc None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:04.530441 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 190/385] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jul 21 22:35:04 2022] GET /identity/v3/groups/5f723cf655a24fbfbf12b2dc2aa3e0c7 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:04.540492 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57803bf9-e67e-47b7-b5da-ae8e6f5efe3d None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:04.680207 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57803bf9-e67e-47b7-b5da-ae8e6f5efe3d None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:04.683880 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:04.726061 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1486fbb-a01b-4c7b-878d-9c16186c9a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:04.726960 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1486fbb-a01b-4c7b-878d-9c16186c9a1f None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:04.727284 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1486fbb-a01b-4c7b-878d-9c16186c9a1f None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:04.727585 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1486fbb-a01b-4c7b-878d-9c16186c9a1f None admin] PATH_INFO: `/v3/projects/6edc20faecff41bf9b7ecc5ac4bc774e` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:04.731502 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1486fbb-a01b-4c7b-878d-9c16186c9a1f None admin] RBAC: Authorizing `identity:get_project(project_id=6edc20faecff41bf9b7ecc5ac4bc774e)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:04.732996 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1486fbb-a01b-4c7b-878d-9c16186c9a1f None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:04.734905 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 196/386] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:35:04 2022] GET /identity/v3/projects/6edc20faecff41bf9b7ecc5ac4bc774e => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:04.742021 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e00accf-1be4-43e0-8085-8a070662b0dc None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:04.787909 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:04.794286 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ddda6-37f9-4d8b-b49e-2e1d656dd3d1 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:04.794517 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ddda6-37f9-4d8b-b49e-2e1d656dd3d1 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:04.794802 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ddda6-37f9-4d8b-b49e-2e1d656dd3d1 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:04.796397 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 197/387] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:04.801720 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce1c31f-82d5-4cf7-a3a9-5fdbf64c2fc3 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:04.802015 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce1c31f-82d5-4cf7-a3a9-5fdbf64c2fc3 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:04.802296 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce1c31f-82d5-4cf7-a3a9-5fdbf64c2fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:04.831851 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-5ce1c31f-82d5-4cf7-a3a9-5fdbf64c2fc3 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:04.833173 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-5ce1c31f-82d5-4cf7-a3a9-5fdbf64c2fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:04.871942 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e00accf-1be4-43e0-8085-8a070662b0dc None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:04.914208 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375dfa9a-48db-457c-8493-50b4910c8725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:04.916219 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375dfa9a-48db-457c-8493-50b4910c8725 None admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:04.916722 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375dfa9a-48db-457c-8493-50b4910c8725 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:04.917389 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375dfa9a-48db-457c-8493-50b4910c8725 None admin] PATH_INFO: `/v3/projects/6edc20faecff41bf9b7ecc5ac4bc774e/groups/5f723cf655a24fbfbf12b2dc2aa3e0c7/roles/b1849c34fc37410098b94ff687ab0055` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:04.920333 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375dfa9a-48db-457c-8493-50b4910c8725 None admin] RBAC: Authorizing `identity:create_grant(project_id=6edc20faecff41bf9b7ecc5ac4bc774e, group_id=5f723cf655a24fbfbf12b2dc2aa3e0c7, role_id=b1849c34fc37410098b94ff687ab0055)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:04.922278 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375dfa9a-48db-457c-8493-50b4910c8725 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:04.956625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 198/388] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:35:04.978750 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce1c31f-82d5-4cf7-a3a9-5fdbf64c2fc3 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:05.045466 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 191/389] 10.222.0.52 () {58 vars in 1442 bytes} [Thu Jul 21 22:35:04 2022] PUT /identity/v3/projects/6edc20faecff41bf9b7ecc5ac4bc774e/groups/5f723cf655a24fbfbf12b2dc2aa3e0c7/roles/b1849c34fc37410098b94ff687ab0055 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:05.094245 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:05.101982 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce1c31f-82d5-4cf7-a3a9-5fdbf64c2fc3 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:05.143648 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef04100-b6ba-4796-997a-00117854e041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:05.144453 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef04100-b6ba-4796-997a-00117854e041 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:05.144731 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef04100-b6ba-4796-997a-00117854e041 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:05.145006 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef04100-b6ba-4796-997a-00117854e041 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:05.156343 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef04100-b6ba-4796-997a-00117854e041 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:35:05.168406 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-5ef04100-b6ba-4796-997a-00117854e041 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:35:05.168406 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:05.168406 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:05.168406 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:05.168406 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:05.168406 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:05.168406 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:05.168406 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:05.168406 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:05.168406 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:05.168406 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:05.168406 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:05.168406 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:35:05.168406 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:35:05.168406 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:35:05.168406 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:35:05.169387 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:35:05.169387 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:35:05.169387 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:35:05.169387 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:35:05.169387 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:05.169387 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:05.169387 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:05.169387 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:05.169387 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:05.169387 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:05.169387 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:05.169387 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:05.169387 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:05.169387 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:05.169387 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:05.169387 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:05.170190 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:05.170190 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:05.170190 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:05.170190 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:05.170190 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:35:05.170190 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:35:05.170190 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:35:05.170190 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:35:05.170190 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:35:05.170190 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:35:05.170190 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:35:05.170190 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:05.170190 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 199/390] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:35:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:35:05.177684 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375dfa9a-48db-457c-8493-50b4910c8725 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:05.295444 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:05.313189 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375dfa9a-48db-457c-8493-50b4910c8725 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:05.354842 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da385c76-3e94-4597-b873-236954df652c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:05.355625 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da385c76-3e94-4597-b873-236954df652c None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:05.355791 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da385c76-3e94-4597-b873-236954df652c None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:05.356038 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da385c76-3e94-4597-b873-236954df652c None admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:05.356792 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da385c76-3e94-4597-b873-236954df652c None admin] RBAC: Adding query filter params (name=member) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:05.357115 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da385c76-3e94-4597-b873-236954df652c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:05.358490 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da385c76-3e94-4597-b873-236954df652c None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:05.371704 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 192/391] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 22:35:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:05.377775 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef04100-b6ba-4796-997a-00117854e041 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:05.499423 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef04100-b6ba-4796-997a-00117854e041 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:05.548416 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb5c660-797d-4040-ad53-5981d035319e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:05.549294 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb5c660-797d-4040-ad53-5981d035319e None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:05.549574 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb5c660-797d-4040-ad53-5981d035319e None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:05.549843 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb5c660-797d-4040-ad53-5981d035319e None admin] PATH_INFO: `/v3/groups/5f723cf655a24fbfbf12b2dc2aa3e0c7` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:05.551046 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb5c660-797d-4040-ad53-5981d035319e None admin] RBAC: Authorizing `identity:get_group(group_id=5f723cf655a24fbfbf12b2dc2aa3e0c7)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:05.552511 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb5c660-797d-4040-ad53-5981d035319e None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:05.554325 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 200/392] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jul 21 22:35:05 2022] GET /identity/v3/groups/5f723cf655a24fbfbf12b2dc2aa3e0c7 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:05.560315 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da385c76-3e94-4597-b873-236954df652c None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:05.589675 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4429976-0418-4dda-bd66-380040d35131 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:05.590050 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4429976-0418-4dda-bd66-380040d35131 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:05.590204 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4429976-0418-4dda-bd66-380040d35131 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:05.591733 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 201/393] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:05.600756 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda1ab2-24e6-47ac-a702-85092c2adf9b None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:05.601083 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda1ab2-24e6-47ac-a702-85092c2adf9b None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:05.601290 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda1ab2-24e6-47ac-a702-85092c2adf9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:05.623995 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-4fda1ab2-24e6-47ac-a702-85092c2adf9b None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:05.625776 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-4fda1ab2-24e6-47ac-a702-85092c2adf9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:05.659159 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:05.716406 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 202/394] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:35:05.720779 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da385c76-3e94-4597-b873-236954df652c None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:05.746042 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda1ab2-24e6-47ac-a702-85092c2adf9b None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:05.761509 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098486bd-f775-49da-9cc9-4b2fa4ff0bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:05.763372 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098486bd-f775-49da-9cc9-4b2fa4ff0bd1 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:05.763914 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098486bd-f775-49da-9cc9-4b2fa4ff0bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:05.764412 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098486bd-f775-49da-9cc9-4b2fa4ff0bd1 None admin] PATH_INFO: `/v3/projects/a54cbea0067945899d2c8c75b7973660` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:05.766075 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098486bd-f775-49da-9cc9-4b2fa4ff0bd1 None admin] RBAC: Authorizing `identity:get_project(project_id=a54cbea0067945899d2c8c75b7973660)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:05.768346 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098486bd-f775-49da-9cc9-4b2fa4ff0bd1 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:05.770670 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 193/395] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:35:05 2022] GET /identity/v3/projects/a54cbea0067945899d2c8c75b7973660 => generated 302 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:05.777260 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098486bd-f775-49da-9cc9-4b2fa4ff0bd1 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:05.800169 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:05.894577 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098486bd-f775-49da-9cc9-4b2fa4ff0bd1 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:05.933048 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f36fe0-02fa-4ede-b315-0ccf68b2723a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:05.934170 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f36fe0-02fa-4ede-b315-0ccf68b2723a None admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:05.934553 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f36fe0-02fa-4ede-b315-0ccf68b2723a None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:05.934966 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f36fe0-02fa-4ede-b315-0ccf68b2723a None admin] PATH_INFO: `/v3/projects/a54cbea0067945899d2c8c75b7973660/groups/5f723cf655a24fbfbf12b2dc2aa3e0c7/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:05.937524 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f36fe0-02fa-4ede-b315-0ccf68b2723a None admin] RBAC: Authorizing `identity:create_grant(project_id=a54cbea0067945899d2c8c75b7973660, group_id=5f723cf655a24fbfbf12b2dc2aa3e0c7, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:05.939392 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f36fe0-02fa-4ede-b315-0ccf68b2723a None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:05.963804 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda1ab2-24e6-47ac-a702-85092c2adf9b None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:06.001422 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e79f40d-dfa5-4ef5-b543-e24f03a0f023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:06.002469 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e79f40d-dfa5-4ef5-b543-e24f03a0f023 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:06.002931 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e79f40d-dfa5-4ef5-b543-e24f03a0f023 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:06.003340 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e79f40d-dfa5-4ef5-b543-e24f03a0f023 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:06.015370 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e79f40d-dfa5-4ef5-b543-e24f03a0f023 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:35:06.022039 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 194/396] 10.222.0.52 () {58 vars in 1442 bytes} [Thu Jul 21 22:35:05 2022] PUT /identity/v3/projects/a54cbea0067945899d2c8c75b7973660/groups/5f723cf655a24fbfbf12b2dc2aa3e0c7/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:35:06.023967 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-2e79f40d-dfa5-4ef5-b543-e24f03a0f023 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:35:06.023967 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:06.023967 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:06.023967 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:06.023967 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:06.023967 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:06.023967 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:06.023967 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:06.023967 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:06.023967 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:06.023967 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:06.023967 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:06.023967 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:35:06.023967 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:35:06.023967 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:35:06.023967 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:35:06.025120 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:35:06.025120 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:35:06.025120 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:35:06.025120 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:35:06.025120 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:06.025120 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:06.025120 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:06.025120 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:06.025120 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:06.025120 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:06.025120 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:06.025120 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:06.025120 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:06.025120 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:06.025120 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:06.025120 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:06.026128 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:06.026128 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:06.026128 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:06.026128 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:06.026128 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:35:06.026128 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:35:06.026128 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:35:06.026128 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:35:06.026128 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:35:06.026128 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:35:06.026128 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:35:06.026128 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:06.028724 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 203/397] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:35:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:35:06.034696 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f36fe0-02fa-4ede-b315-0ccf68b2723a None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:06.174855 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:06.208434 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f36fe0-02fa-4ede-b315-0ccf68b2723a None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:06.269853 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cd934d-f1f9-46aa-85c1-d1994c9d056e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:06.270527 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd934d-f1f9-46aa-85c1-d1994c9d056e None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:06.270847 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd934d-f1f9-46aa-85c1-d1994c9d056e None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:06.271121 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd934d-f1f9-46aa-85c1-d1994c9d056e None admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:06.271864 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cd934d-f1f9-46aa-85c1-d1994c9d056e None admin] RBAC: Adding query filter params (name=member) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:06.272166 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cd934d-f1f9-46aa-85c1-d1994c9d056e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:06.273538 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cd934d-f1f9-46aa-85c1-d1994c9d056e None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:06.285799 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 195/398] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 22:35:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:06.291569 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e79f40d-dfa5-4ef5-b543-e24f03a0f023 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:06.312077 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:06.405289 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e79f40d-dfa5-4ef5-b543-e24f03a0f023 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:06.441381 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74016fba-cb48-43eb-8685-2cfe512c344c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:06.442156 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74016fba-cb48-43eb-8685-2cfe512c344c None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:06.442432 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74016fba-cb48-43eb-8685-2cfe512c344c None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:06.442702 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74016fba-cb48-43eb-8685-2cfe512c344c None admin] PATH_INFO: `/v3/groups/5f723cf655a24fbfbf12b2dc2aa3e0c7` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:06.443904 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74016fba-cb48-43eb-8685-2cfe512c344c None admin] RBAC: Authorizing `identity:get_group(group_id=5f723cf655a24fbfbf12b2dc2aa3e0c7)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:06.445324 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74016fba-cb48-43eb-8685-2cfe512c344c None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:06.447839 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 204/399] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jul 21 22:35:06 2022] GET /identity/v3/groups/5f723cf655a24fbfbf12b2dc2aa3e0c7 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:35:06.455866 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cd934d-f1f9-46aa-85c1-d1994c9d056e None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:06.591979 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cd934d-f1f9-46aa-85c1-d1994c9d056e None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:06.634059 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bea153e-48bd-4302-8bff-442c3835bb3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:06.635310 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea153e-48bd-4302-8bff-442c3835bb3a None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:06.635798 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea153e-48bd-4302-8bff-442c3835bb3a None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:06.636270 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea153e-48bd-4302-8bff-442c3835bb3a None admin] PATH_INFO: `/v3/projects/6edc20faecff41bf9b7ecc5ac4bc774e` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:06.643509 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bea153e-48bd-4302-8bff-442c3835bb3a None admin] RBAC: Authorizing `identity:get_project(project_id=6edc20faecff41bf9b7ecc5ac4bc774e)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:06.645221 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bea153e-48bd-4302-8bff-442c3835bb3a None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:06.647203 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 196/400] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:35:06 2022] GET /identity/v3/projects/6edc20faecff41bf9b7ecc5ac4bc774e => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:06.653642 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74016fba-cb48-43eb-8685-2cfe512c344c None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:06.740353 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01bd3b-9f8b-4f4f-b6a1-ade119a0efe6 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:06.741410 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01bd3b-9f8b-4f4f-b6a1-ade119a0efe6 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:06.742197 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01bd3b-9f8b-4f4f-b6a1-ade119a0efe6 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:06.744274 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 197/401] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:06.749671 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c218ad80-d4d3-441a-a67a-e269cded2fdf None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:06.749954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c218ad80-d4d3-441a-a67a-e269cded2fdf None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:06.750231 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c218ad80-d4d3-441a-a67a-e269cded2fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:06.776912 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-c218ad80-d4d3-441a-a67a-e269cded2fdf None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:06.778122 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-c218ad80-d4d3-441a-a67a-e269cded2fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:06.781514 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:06.794125 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:06.819563 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74016fba-cb48-43eb-8685-2cfe512c344c None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:06.858524 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de32eef-a2ec-42c3-9c78-c90398f109cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:06.859448 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de32eef-a2ec-42c3-9c78-c90398f109cb None admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:06.859731 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de32eef-a2ec-42c3-9c78-c90398f109cb None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:06.860008 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de32eef-a2ec-42c3-9c78-c90398f109cb None admin] PATH_INFO: `/v3/projects/6edc20faecff41bf9b7ecc5ac4bc774e/groups/5f723cf655a24fbfbf12b2dc2aa3e0c7/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:06.862141 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de32eef-a2ec-42c3-9c78-c90398f109cb None admin] RBAC: Authorizing `identity:create_grant(project_id=6edc20faecff41bf9b7ecc5ac4bc774e, group_id=5f723cf655a24fbfbf12b2dc2aa3e0c7, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:06.863601 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de32eef-a2ec-42c3-9c78-c90398f109cb None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:06.882169 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 198/402] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:35:06.883614 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f385c2e-0564-4b27-b3fb-b7411008c040 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:06.883910 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f385c2e-0564-4b27-b3fb-b7411008c040 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:06.884187 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f385c2e-0564-4b27-b3fb-b7411008c040 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:06.889591 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 199/403] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:06.890891 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da41765-9f16-4ba6-af4e-05fcec050f16 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:06.891177 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da41765-9f16-4ba6-af4e-05fcec050f16 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:06.891457 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da41765-9f16-4ba6-af4e-05fcec050f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:06.908801 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-9da41765-9f16-4ba6-af4e-05fcec050f16 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:06.909999 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-9da41765-9f16-4ba6-af4e-05fcec050f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:06.963949 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 205/404] 10.222.0.52 () {58 vars in 1442 bytes} [Thu Jul 21 22:35:06 2022] PUT /identity/v3/projects/6edc20faecff41bf9b7ecc5ac4bc774e/groups/5f723cf655a24fbfbf12b2dc2aa3e0c7/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:06.967683 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de32eef-a2ec-42c3-9c78-c90398f109cb None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:07.003569 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 200/405] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:35:07.055764 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da41765-9f16-4ba6-af4e-05fcec050f16 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:07.161219 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de32eef-a2ec-42c3-9c78-c90398f109cb None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:07.179430 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:07.215513 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaa9066-ffaf-46ac-820f-be9c69e79a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:07.216337 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaa9066-ffaf-46ac-820f-be9c69e79a8b None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:07.216623 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaa9066-ffaf-46ac-820f-be9c69e79a8b None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:07.216902 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaa9066-ffaf-46ac-820f-be9c69e79a8b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:07.220455 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da41765-9f16-4ba6-af4e-05fcec050f16 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:07.228885 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaa9066-ffaf-46ac-820f-be9c69e79a8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:35:07.237312 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-eaaa9066-ffaf-46ac-820f-be9c69e79a8b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:35:07.237312 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:07.237312 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:07.237312 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:07.237312 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:07.237312 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:07.237312 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:07.237312 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:07.237312 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:07.237312 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:07.237312 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:07.237312 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:07.237312 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:35:07.237312 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:35:07.237312 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:35:07.237312 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:35:07.238222 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:35:07.238222 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:35:07.238222 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:35:07.238222 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:35:07.238222 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:07.238222 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:07.238222 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:07.238222 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:07.238222 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:07.238222 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:07.238222 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:07.238222 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:07.238222 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:07.238222 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:07.238222 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:07.238222 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:07.239080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:07.239080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:07.239080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:07.239080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:07.239080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:35:07.239080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:35:07.239080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:35:07.239080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:35:07.239080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:35:07.239080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:35:07.239080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:35:07.239080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:07.239080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 206/406] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 21 22:35:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:07.244385 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaa9066-ffaf-46ac-820f-be9c69e79a8b None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:07.253752 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:07.262649 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaba70c9-0ecc-4033-923b-901806617919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:07.268397 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaba70c9-0ecc-4033-923b-901806617919 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:07.268397 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaba70c9-0ecc-4033-923b-901806617919 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:07.268397 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaba70c9-0ecc-4033-923b-901806617919 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:07.276096 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaba70c9-0ecc-4033-923b-901806617919 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:35:07.284281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-eaba70c9-0ecc-4033-923b-901806617919 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:35:07.284281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:07.284281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:07.284281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:07.284281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:07.284281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:07.284281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:07.284281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:07.284281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:07.284281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:07.284281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:07.284281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:07.284281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:35:07.284281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:35:07.284281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:35:07.284281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:35:07.285156 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:35:07.285156 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:35:07.285156 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:35:07.285156 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:35:07.285156 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:07.285156 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:07.285156 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:07.285156 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:07.285156 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:07.285156 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:07.285156 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:07.285156 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:07.285156 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:07.285156 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:07.285156 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:07.285156 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:07.285941 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:07.285941 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:07.285941 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:07.285941 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:07.285941 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:35:07.285941 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:35:07.285941 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:35:07.285941 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:35:07.285941 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:35:07.285941 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:35:07.285941 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:35:07.285941 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:07.288572 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 201/407] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 22:35:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:35:07.294427 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaba70c9-0ecc-4033-923b-901806617919 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:07.353911 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaa9066-ffaf-46ac-820f-be9c69e79a8b None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:07.413635 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaba70c9-0ecc-4033-923b-901806617919 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:07.417900 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791ed432-9017-444c-9e2e-6811f5a09710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:07.419238 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ed432-9017-444c-9e2e-6811f5a09710 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:07.419742 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ed432-9017-444c-9e2e-6811f5a09710 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:07.420213 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ed432-9017-444c-9e2e-6811f5a09710 None admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:07.421164 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791ed432-9017-444c-9e2e-6811f5a09710 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:07.421667 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791ed432-9017-444c-9e2e-6811f5a09710 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:07.431400 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791ed432-9017-444c-9e2e-6811f5a09710 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:07.447149 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:07.450791 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 207/408] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:35:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:35:07.459191 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254123b9-2f5a-410c-be14-79beade987a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:07.459957 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791ed432-9017-444c-9e2e-6811f5a09710 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:07.461045 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254123b9-2f5a-410c-be14-79beade987a3 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:07.461619 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254123b9-2f5a-410c-be14-79beade987a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:07.468660 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254123b9-2f5a-410c-be14-79beade987a3 None admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:07.468660 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254123b9-2f5a-410c-be14-79beade987a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:07.468660 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254123b9-2f5a-410c-be14-79beade987a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:07.468660 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254123b9-2f5a-410c-be14-79beade987a3 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:07.480881 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 202/409] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:35:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:35:07.490845 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254123b9-2f5a-410c-be14-79beade987a3 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:07.601841 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254123b9-2f5a-410c-be14-79beade987a3 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:07.641446 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c645f2-d4e0-4791-9f0b-4fe881af8300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:07.642317 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c645f2-d4e0-4791-9f0b-4fe881af8300 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:07.642603 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c645f2-d4e0-4791-9f0b-4fe881af8300 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:07.642910 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c645f2-d4e0-4791-9f0b-4fe881af8300 None admin] PATH_INFO: `/v3/groups/d77a5b66867941989c313e3a336978b3` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:07.644508 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c645f2-d4e0-4791-9f0b-4fe881af8300 None admin] RBAC: Authorizing `identity:get_group(group_id=d77a5b66867941989c313e3a336978b3)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:07.646210 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c645f2-d4e0-4791-9f0b-4fe881af8300 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:07.648498 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 203/410] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jul 21 22:35:07 2022] GET /identity/v3/groups/d77a5b66867941989c313e3a336978b3 => generated 229 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:07.654836 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c645f2-d4e0-4791-9f0b-4fe881af8300 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:07.668183 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:07.722629 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:07.749633 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791ed432-9017-444c-9e2e-6811f5a09710 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:07.772520 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c645f2-d4e0-4791-9f0b-4fe881af8300 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:07.812769 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa409d5a-1159-406f-bbdc-f5f1307b78b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:07.813637 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa409d5a-1159-406f-bbdc-f5f1307b78b5 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:07.813923 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa409d5a-1159-406f-bbdc-f5f1307b78b5 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:07.814200 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa409d5a-1159-406f-bbdc-f5f1307b78b5 None admin] PATH_INFO: `/v3/projects/d47c52ce443d40eaa4e635852fb964cd` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:07.815655 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa409d5a-1159-406f-bbdc-f5f1307b78b5 None admin] RBAC: Authorizing `identity:get_project(project_id=d47c52ce443d40eaa4e635852fb964cd)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:07.817539 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa409d5a-1159-406f-bbdc-f5f1307b78b5 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:07.820198 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 204/411] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:35:07 2022] GET /identity/v3/projects/d47c52ce443d40eaa4e635852fb964cd => generated 344 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:07.826518 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa409d5a-1159-406f-bbdc-f5f1307b78b5 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:07.842955 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5397bc-1052-4320-b438-871e4e29a687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:07.844151 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5397bc-1052-4320-b438-871e4e29a687 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:07.844650 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5397bc-1052-4320-b438-871e4e29a687 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:07.845136 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5397bc-1052-4320-b438-871e4e29a687 None admin] PATH_INFO: `/v3/groups/5f723cf655a24fbfbf12b2dc2aa3e0c7` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:07.851575 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5397bc-1052-4320-b438-871e4e29a687 None admin] RBAC: Authorizing `identity:get_group(group_id=5f723cf655a24fbfbf12b2dc2aa3e0c7)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:07.854130 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5397bc-1052-4320-b438-871e4e29a687 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:07.856322 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 208/412] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jul 21 22:35:07 2022] GET /identity/v3/groups/5f723cf655a24fbfbf12b2dc2aa3e0c7 => generated 226 bytes in 400 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:35:07.863647 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5397bc-1052-4320-b438-871e4e29a687 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:07.942671 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa409d5a-1159-406f-bbdc-f5f1307b78b5 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:07.992883 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:08.010709 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e32c5-8d09-4e7f-bfe0-0f9d4939964a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:08.011619 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e32c5-8d09-4e7f-bfe0-0f9d4939964a None admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:08.011897 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e32c5-8d09-4e7f-bfe0-0f9d4939964a None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:08.012166 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e32c5-8d09-4e7f-bfe0-0f9d4939964a None admin] PATH_INFO: `/v3/projects/d47c52ce443d40eaa4e635852fb964cd/groups/d77a5b66867941989c313e3a336978b3/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:08.015430 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1e32c5-8d09-4e7f-bfe0-0f9d4939964a None admin] RBAC: Authorizing `identity:create_grant(project_id=d47c52ce443d40eaa4e635852fb964cd, group_id=d77a5b66867941989c313e3a336978b3, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:08.016878 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1e32c5-8d09-4e7f-bfe0-0f9d4939964a None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:08.026155 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5397bc-1052-4320-b438-871e4e29a687 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:08.061493 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfba04aa-9d56-48f2-bf81-c2d62ab1cd8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:08.062368 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba04aa-9d56-48f2-bf81-c2d62ab1cd8f None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:08.062646 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba04aa-9d56-48f2-bf81-c2d62ab1cd8f None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:08.062943 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba04aa-9d56-48f2-bf81-c2d62ab1cd8f None admin] PATH_INFO: `/v3/projects/a54cbea0067945899d2c8c75b7973660` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:08.064140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfba04aa-9d56-48f2-bf81-c2d62ab1cd8f None admin] RBAC: Authorizing `identity:get_project(project_id=a54cbea0067945899d2c8c75b7973660)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:08.065660 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfba04aa-9d56-48f2-bf81-c2d62ab1cd8f None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:08.067757 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 209/413] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:35:07 2022] GET /identity/v3/projects/a54cbea0067945899d2c8c75b7973660 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:08.073980 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfba04aa-9d56-48f2-bf81-c2d62ab1cd8f None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:08.078126 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:08.113104 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 205/414] 10.222.0.52 () {58 vars in 1442 bytes} [Thu Jul 21 22:35:07 2022] PUT /identity/v3/projects/d47c52ce443d40eaa4e635852fb964cd/groups/d77a5b66867941989c313e3a336978b3/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:08.203639 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfba04aa-9d56-48f2-bf81-c2d62ab1cd8f None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:08.248156 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6ec852-76cb-4297-b2ab-f62fb3b3d6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:08.249518 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ec852-76cb-4297-b2ab-f62fb3b3d6bf None admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:08.250048 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ec852-76cb-4297-b2ab-f62fb3b3d6bf None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:08.250527 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ec852-76cb-4297-b2ab-f62fb3b3d6bf None admin] PATH_INFO: `/v3/projects/a54cbea0067945899d2c8c75b7973660/groups/5f723cf655a24fbfbf12b2dc2aa3e0c7/roles/b1849c34fc37410098b94ff687ab0055` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:08.253086 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6ec852-76cb-4297-b2ab-f62fb3b3d6bf None admin] RBAC: Authorizing `identity:create_grant(project_id=a54cbea0067945899d2c8c75b7973660, group_id=5f723cf655a24fbfbf12b2dc2aa3e0c7, role_id=b1849c34fc37410098b94ff687ab0055)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:08.254845 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6ec852-76cb-4297-b2ab-f62fb3b3d6bf None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:08.271455 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988bc29-85a7-4b5e-b4a6-c885e833f8ee None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:08.272062 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988bc29-85a7-4b5e-b4a6-c885e833f8ee None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:08.272504 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988bc29-85a7-4b5e-b4a6-c885e833f8ee None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:08.277190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 206/415] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:35:08.281693 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267761b0-e042-40fa-a13b-dcd46c46afe2 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:08.282255 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267761b0-e042-40fa-a13b-dcd46c46afe2 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:08.282830 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267761b0-e042-40fa-a13b-dcd46c46afe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:08.302744 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-267761b0-e042-40fa-a13b-dcd46c46afe2 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:08.304488 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-267761b0-e042-40fa-a13b-dcd46c46afe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:08.359852 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 210/416] 10.222.0.52 () {58 vars in 1442 bytes} [Thu Jul 21 22:35:08 2022] PUT /identity/v3/projects/a54cbea0067945899d2c8c75b7973660/groups/5f723cf655a24fbfbf12b2dc2aa3e0c7/roles/b1849c34fc37410098b94ff687ab0055 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:35:08.397763 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 207/417] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:35:08.441558 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6ec852-76cb-4297-b2ab-f62fb3b3d6bf None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:08.493819 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6268825-4082-4de0-9a04-61000bd5119e None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:08.494317 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6268825-4082-4de0-9a04-61000bd5119e None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:08.494415 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6268825-4082-4de0-9a04-61000bd5119e None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:08.496055 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 208/418] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:08.501564 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391538ec-943b-4502-975e-108f44e28a50 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:08.501861 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391538ec-943b-4502-975e-108f44e28a50 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:08.502126 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391538ec-943b-4502-975e-108f44e28a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:08.521080 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-391538ec-943b-4502-975e-108f44e28a50 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:08.522493 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-391538ec-943b-4502-975e-108f44e28a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:08.558517 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:08.616472 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6ec852-76cb-4297-b2ab-f62fb3b3d6bf None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:08.637660 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 209/419] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:35:08.660364 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391538ec-943b-4502-975e-108f44e28a50 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:08.674082 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe758044-0e65-442c-b1ae-445f09f4a6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:08.675270 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe758044-0e65-442c-b1ae-445f09f4a6f1 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:08.675554 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe758044-0e65-442c-b1ae-445f09f4a6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:08.675826 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe758044-0e65-442c-b1ae-445f09f4a6f1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:08.690312 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe758044-0e65-442c-b1ae-445f09f4a6f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:35:08.701298 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-fe758044-0e65-442c-b1ae-445f09f4a6f1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:35:08.701298 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:08.701298 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:08.701298 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:08.701298 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:08.701298 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:08.701298 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:08.701298 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:08.701298 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:08.701298 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:08.701298 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:08.701298 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:08.701298 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:35:08.701298 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:35:08.701298 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:35:08.701298 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:35:08.702162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:35:08.702162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:35:08.702162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:35:08.702162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:35:08.702162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:08.702162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:08.702162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:08.702162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:08.702162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:08.702162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:08.702162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:08.702162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:08.702162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:08.702162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:08.702162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:08.702162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:08.707456 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:08.707456 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:08.707456 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:08.707456 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:08.707456 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:35:08.707456 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:35:08.707456 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:35:08.707456 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:35:08.707456 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:35:08.707456 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:35:08.707456 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:35:08.707456 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:08.707456 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 211/420] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 21 22:35:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:08.711306 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe758044-0e65-442c-b1ae-445f09f4a6f1 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:08.731180 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:08.784933 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391538ec-943b-4502-975e-108f44e28a50 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:08.822814 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac1f9d1-3a6b-4c60-8c3c-7b7043d6b239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:08.823631 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac1f9d1-3a6b-4c60-8c3c-7b7043d6b239 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:08.823918 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac1f9d1-3a6b-4c60-8c3c-7b7043d6b239 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:08.824197 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac1f9d1-3a6b-4c60-8c3c-7b7043d6b239 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:08.835275 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac1f9d1-3a6b-4c60-8c3c-7b7043d6b239 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:35:08.848160 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-2ac1f9d1-3a6b-4c60-8c3c-7b7043d6b239 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:35:08.848160 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:08.848160 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:08.848160 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:08.848160 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:08.848160 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:08.848160 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:08.848160 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:08.848160 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:08.848160 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:08.848160 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:08.848160 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:08.848160 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:35:08.848160 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:35:08.848160 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:35:08.848160 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:35:08.849075 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:35:08.849075 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:35:08.849075 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:35:08.849075 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:35:08.849075 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:08.849075 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:08.849075 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:08.849075 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:08.849075 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:08.849075 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:08.849075 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:08.849075 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:08.849075 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:08.849075 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:08.849075 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:08.849075 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:08.849884 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:08.849884 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:08.849884 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:08.849884 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:08.849884 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:35:08.849884 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:35:08.849884 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:35:08.849884 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:35:08.849884 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:35:08.849884 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:35:08.849884 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:35:08.849884 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:08.849884 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 210/421] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:35:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:35:08.856460 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac1f9d1-3a6b-4c60-8c3c-7b7043d6b239 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:08.969864 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac1f9d1-3a6b-4c60-8c3c-7b7043d6b239 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:08.983125 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:08.987638 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe758044-0e65-442c-b1ae-445f09f4a6f1 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:09.038126 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e69086-5038-4e07-adfc-8358ef586a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:09.038857 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e69086-5038-4e07-adfc-8358ef586a3f None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:09.039119 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e69086-5038-4e07-adfc-8358ef586a3f None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:09.040975 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e69086-5038-4e07-adfc-8358ef586a3f None admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:09.041761 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e69086-5038-4e07-adfc-8358ef586a3f None admin] RBAC: Adding query filter params (name=member) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:09.042077 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e69086-5038-4e07-adfc-8358ef586a3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:09.043187 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:09.043512 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e69086-5038-4e07-adfc-8358ef586a3f None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:09.057896 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ba017-c8b5-434c-befd-a828a2be09a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:09.059583 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ba017-c8b5-434c-befd-a828a2be09a7 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:09.059583 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ba017-c8b5-434c-befd-a828a2be09a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:09.059583 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ba017-c8b5-434c-befd-a828a2be09a7 None admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:09.059882 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9ba017-c8b5-434c-befd-a828a2be09a7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:09.060198 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9ba017-c8b5-434c-befd-a828a2be09a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:09.061575 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9ba017-c8b5-434c-befd-a828a2be09a7 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:09.065085 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 211/422] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 22:35:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:09.075786 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e69086-5038-4e07-adfc-8358ef586a3f None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:09.077949 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 212/423] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:35:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:09.083320 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ba017-c8b5-434c-befd-a828a2be09a7 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:09.240480 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ba017-c8b5-434c-befd-a828a2be09a7 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:09.261887 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e69086-5038-4e07-adfc-8358ef586a3f None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:09.310802 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647e9566-6281-4ca8-aa35-f106c1ff0db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:09.311627 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647e9566-6281-4ca8-aa35-f106c1ff0db0 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:09.311910 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647e9566-6281-4ca8-aa35-f106c1ff0db0 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:09.312187 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647e9566-6281-4ca8-aa35-f106c1ff0db0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:09.321127 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6f2c14-0d6f-4f4f-85ef-cd51f6982455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:09.321781 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f2c14-0d6f-4f4f-85ef-cd51f6982455 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:09.322062 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f2c14-0d6f-4f4f-85ef-cd51f6982455 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:09.322587 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f2c14-0d6f-4f4f-85ef-cd51f6982455 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:09.389442 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:09.415944 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:09.439694 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647e9566-6281-4ca8-aa35-f106c1ff0db0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:09.441166 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647e9566-6281-4ca8-aa35-f106c1ff0db0 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:09.453875 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6f2c14-0d6f-4f4f-85ef-cd51f6982455 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:09.455353 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6f2c14-0d6f-4f4f-85ef-cd51f6982455 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:09.593108 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 212/424] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:35:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:35:09.595371 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4da588-54a4-471b-93aa-da067f18d92b None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:09.595848 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4da588-54a4-471b-93aa-da067f18d92b None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:09.596284 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4da588-54a4-471b-93aa-da067f18d92b None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:09.608237 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 213/425] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:09.608237 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4da588-54a4-471b-93aa-da067f18d92b None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:09.608542 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 213/426] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:35:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:35:09.608542 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec2bba-e354-4997-bbf0-441be6c5c342 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:09.608542 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec2bba-e354-4997-bbf0-441be6c5c342 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:09.608542 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec2bba-e354-4997-bbf0-441be6c5c342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:09.622954 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-c6ec2bba-e354-4997-bbf0-441be6c5c342 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:09.624167 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-c6ec2bba-e354-4997-bbf0-441be6c5c342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:09.701375 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 214/427] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:35:09.702394 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ec2bba-e354-4997-bbf0-441be6c5c342 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:09.716477 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:09.724280 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:09.768126 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4da588-54a4-471b-93aa-da067f18d92b None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:09.806972 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5139add9-f9f9-417b-a7d8-99b580839679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:09.807835 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5139add9-f9f9-417b-a7d8-99b580839679 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:09.808120 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5139add9-f9f9-417b-a7d8-99b580839679 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:09.808397 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5139add9-f9f9-417b-a7d8-99b580839679 None admin] PATH_INFO: `/v3/projects/a54cbea0067945899d2c8c75b7973660` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:09.809588 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5139add9-f9f9-417b-a7d8-99b580839679 None admin] RBAC: Authorizing `identity:get_project(project_id=a54cbea0067945899d2c8c75b7973660)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:09.811030 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5139add9-f9f9-417b-a7d8-99b580839679 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:09.817538 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 214/428] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:35:09 2022] GET /identity/v3/projects/a54cbea0067945899d2c8c75b7973660 => generated 302 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:09.818340 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5139add9-f9f9-417b-a7d8-99b580839679 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:09.819729 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ec2bba-e354-4997-bbf0-441be6c5c342 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:09.856909 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86649f5-d3bb-44bf-b16f-9a949c2a776f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:09.857767 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86649f5-d3bb-44bf-b16f-9a949c2a776f None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:09.858045 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86649f5-d3bb-44bf-b16f-9a949c2a776f None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:09.858318 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86649f5-d3bb-44bf-b16f-9a949c2a776f None admin] PATH_INFO: `/v3/projects/6edc20faecff41bf9b7ecc5ac4bc774e` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:09.859649 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86649f5-d3bb-44bf-b16f-9a949c2a776f None admin] RBAC: Authorizing `identity:get_project(project_id=6edc20faecff41bf9b7ecc5ac4bc774e)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:09.861075 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86649f5-d3bb-44bf-b16f-9a949c2a776f None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:09.866973 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 215/429] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:35:09 2022] GET /identity/v3/projects/6edc20faecff41bf9b7ecc5ac4bc774e => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:09.868392 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86649f5-d3bb-44bf-b16f-9a949c2a776f None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:09.921248 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5139add9-f9f9-417b-a7d8-99b580839679 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:09.956856 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14760a28-2a05-41e5-a1a6-fbb03eb1937f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:09.957663 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14760a28-2a05-41e5-a1a6-fbb03eb1937f None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:09.957944 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14760a28-2a05-41e5-a1a6-fbb03eb1937f None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:09.958216 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14760a28-2a05-41e5-a1a6-fbb03eb1937f None admin] PATH_INFO: `/v3/roles/member` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:09.959436 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:09.961318 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:09.969944 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14760a28-2a05-41e5-a1a6-fbb03eb1937f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:35:09.976423 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86649f5-d3bb-44bf-b16f-9a949c2a776f None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:09.977893 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-14760a28-2a05-41e5-a1a6-fbb03eb1937f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:35:09.977893 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:09.977893 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:09.977893 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:09.977893 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:09.977893 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:09.977893 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:09.977893 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:09.977893 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:09.977893 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:09.977893 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:09.977893 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:09.977893 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:35:09.977893 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:35:09.977893 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:35:09.977893 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:35:09.981196 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:35:09.981196 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:35:09.981196 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:35:09.981196 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:35:09.981196 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:09.981196 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:09.981196 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:09.981196 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:09.981196 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:09.981196 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:09.981196 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:09.981196 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:09.981196 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:09.981196 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:09.981196 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:09.981196 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:09.982145 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:09.982145 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:09.982145 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:09.982145 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:09.982145 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:35:09.982145 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:35:09.982145 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:35:09.982145 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:35:09.982145 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:35:09.982145 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:35:09.982145 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 22:35:09.982145 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:09.982145 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 215/430] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:35:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:09.982145 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14760a28-2a05-41e5-a1a6-fbb03eb1937f None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:10.016222 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac2c87c-7130-4c97-8211-5ce8503c5823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:10.017296 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2c87c-7130-4c97-8211-5ce8503c5823 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:10.017715 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2c87c-7130-4c97-8211-5ce8503c5823 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:10.018202 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2c87c-7130-4c97-8211-5ce8503c5823 None admin] PATH_INFO: `/v3/groups/5f723cf655a24fbfbf12b2dc2aa3e0c7` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:10.019545 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac2c87c-7130-4c97-8211-5ce8503c5823 None admin] RBAC: Authorizing `identity:get_group(group_id=5f723cf655a24fbfbf12b2dc2aa3e0c7)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:10.021120 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac2c87c-7130-4c97-8211-5ce8503c5823 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:10.023168 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 216/431] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jul 21 22:35:09 2022] GET /identity/v3/groups/5f723cf655a24fbfbf12b2dc2aa3e0c7 => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:10.024082 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac2c87c-7130-4c97-8211-5ce8503c5823 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:10.095301 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14760a28-2a05-41e5-a1a6-fbb03eb1937f None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:10.131434 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c198cc52-1741-4f04-9c97-f23f00cc4355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:10.132443 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c198cc52-1741-4f04-9c97-f23f00cc4355 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:10.132882 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c198cc52-1741-4f04-9c97-f23f00cc4355 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:10.133798 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c198cc52-1741-4f04-9c97-f23f00cc4355 None admin] PATH_INFO: `/v3/groups/5f723cf655a24fbfbf12b2dc2aa3e0c7` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:10.135095 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c198cc52-1741-4f04-9c97-f23f00cc4355 None admin] RBAC: Authorizing `identity:get_group(group_id=5f723cf655a24fbfbf12b2dc2aa3e0c7)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:10.136778 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c198cc52-1741-4f04-9c97-f23f00cc4355 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:10.138410 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 216/432] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jul 21 22:35:09 2022] GET /identity/v3/groups/5f723cf655a24fbfbf12b2dc2aa3e0c7 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:10.139479 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c198cc52-1741-4f04-9c97-f23f00cc4355 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:10.145451 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac2c87c-7130-4c97-8211-5ce8503c5823 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:10.184638 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14fcf73-adaa-47a5-b2bc-968a8d47b062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:10.185642 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14fcf73-adaa-47a5-b2bc-968a8d47b062 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:10.186096 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14fcf73-adaa-47a5-b2bc-968a8d47b062 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:10.186561 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14fcf73-adaa-47a5-b2bc-968a8d47b062 None admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:10.187577 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14fcf73-adaa-47a5-b2bc-968a8d47b062 None admin] RBAC: Adding query filter params (name=member) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:10.188106 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14fcf73-adaa-47a5-b2bc-968a8d47b062 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:10.189763 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14fcf73-adaa-47a5-b2bc-968a8d47b062 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:10.199582 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 217/433] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 22:35:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:10.200769 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14fcf73-adaa-47a5-b2bc-968a8d47b062 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:10.207646 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:10.221829 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:10.247753 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c198cc52-1741-4f04-9c97-f23f00cc4355 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:10.285630 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e91f10b-6427-4312-a25e-5b18edc23011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:10.286421 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e91f10b-6427-4312-a25e-5b18edc23011 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:10.286529 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e91f10b-6427-4312-a25e-5b18edc23011 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:10.286821 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e91f10b-6427-4312-a25e-5b18edc23011 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:10.287662 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e91f10b-6427-4312-a25e-5b18edc23011 None admin] RBAC: Adding query filter params (group.id=5f723cf655a24fbfbf12b2dc2aa3e0c7, role.id=53d3bdbe9cdf4d35bc73854a33df8e45, scope.project.id=a54cbea0067945899d2c8c75b7973660) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:10.287971 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e91f10b-6427-4312-a25e-5b18edc23011 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:10.289454 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e91f10b-6427-4312-a25e-5b18edc23011 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:10.303126 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 217/434] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Jul 21 22:35:10 2022] GET /identity/v3/role_assignments?group.id=5f723cf655a24fbfbf12b2dc2aa3e0c7&scope.project.id=a54cbea0067945899d2c8c75b7973660&role.id=53d3bdbe9cdf4d35bc73854a33df8e45 => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:10.304239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e91f10b-6427-4312-a25e-5b18edc23011 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:10.314337 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14fcf73-adaa-47a5-b2bc-968a8d47b062 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:10.356147 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3a380c-e865-40ff-b745-e3f9501c53f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:10.357187 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a380c-e865-40ff-b745-e3f9501c53f0 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:10.357683 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a380c-e865-40ff-b745-e3f9501c53f0 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:10.358172 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a380c-e865-40ff-b745-e3f9501c53f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:10.359229 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3a380c-e865-40ff-b745-e3f9501c53f0 None admin] RBAC: Adding query filter params (group.id=5f723cf655a24fbfbf12b2dc2aa3e0c7, role.id=b1849c34fc37410098b94ff687ab0055, scope.project.id=6edc20faecff41bf9b7ecc5ac4bc774e) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:10.359733 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3a380c-e865-40ff-b745-e3f9501c53f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:10.361395 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3a380c-e865-40ff-b745-e3f9501c53f0 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:10.370326 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 218/435] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Jul 21 22:35:10 2022] GET /identity/v3/role_assignments?group.id=5f723cf655a24fbfbf12b2dc2aa3e0c7&scope.project.id=6edc20faecff41bf9b7ecc5ac4bc774e&role.id=b1849c34fc37410098b94ff687ab0055 => generated 611 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:10.451896 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e91f10b-6427-4312-a25e-5b18edc23011 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:10.484403 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:10.494701 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a99dba-c396-4a21-8b6f-d57994c095c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:10.495491 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a99dba-c396-4a21-8b6f-d57994c095c7 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:10.495776 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a99dba-c396-4a21-8b6f-d57994c095c7 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:10.496055 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a99dba-c396-4a21-8b6f-d57994c095c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:10.536740 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c79e49f-408b-4e7d-990f-3716a51b6712 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:10.537380 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c79e49f-408b-4e7d-990f-3716a51b6712 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:10.537847 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c79e49f-408b-4e7d-990f-3716a51b6712 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:10.539493 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 219/436] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:10.546718 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e4e41f-d012-4138-a30f-f6e02f7cf437 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:10.547408 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e4e41f-d012-4138-a30f-f6e02f7cf437 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:10.547891 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e4e41f-d012-4138-a30f-f6e02f7cf437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:10.578840 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-56e4e41f-d012-4138-a30f-f6e02f7cf437 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:10.580646 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-56e4e41f-d012-4138-a30f-f6e02f7cf437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:10.580948 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a99dba-c396-4a21-8b6f-d57994c095c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:10.582507 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a99dba-c396-4a21-8b6f-d57994c095c7 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:10.668393 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 220/437] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:35:10.695085 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e4e41f-d012-4138-a30f-f6e02f7cf437 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:10.706464 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 218/438] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:35:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:35:10.707593 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:10.711801 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a99dba-c396-4a21-8b6f-d57994c095c7 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:10.741341 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:10.804294 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e4e41f-d012-4138-a30f-f6e02f7cf437 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:10.821531 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a99dba-c396-4a21-8b6f-d57994c095c7 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:10.839873 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7f5d46-0adc-4e00-8df1-957e200ab398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:10.840966 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f5d46-0adc-4e00-8df1-957e200ab398 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:10.841253 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f5d46-0adc-4e00-8df1-957e200ab398 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:10.841530 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f5d46-0adc-4e00-8df1-957e200ab398 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:10.853226 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7f5d46-0adc-4e00-8df1-957e200ab398 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:35:10.857610 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fe3546-18d0-44f1-bb39-446b023dffd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:10.858468 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe3546-18d0-44f1-bb39-446b023dffd7 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:10.858749 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe3546-18d0-44f1-bb39-446b023dffd7 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:10.859047 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe3546-18d0-44f1-bb39-446b023dffd7 None admin] PATH_INFO: `/v3/projects/6edc20faecff41bf9b7ecc5ac4bc774e` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:10.860214 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fe3546-18d0-44f1-bb39-446b023dffd7 None admin] RBAC: Authorizing `identity:get_project(project_id=6edc20faecff41bf9b7ecc5ac4bc774e)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:10.860832 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-7a7f5d46-0adc-4e00-8df1-957e200ab398 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:35:10.860832 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:10.860832 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:10.860832 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:10.860832 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:10.860832 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:10.860832 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:10.860832 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:10.860832 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:10.860832 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:10.860832 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:10.860832 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:10.860832 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:35:10.860832 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:35:10.860832 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:35:10.860832 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:35:10.861744 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:35:10.861744 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:35:10.861744 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:35:10.861744 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:35:10.861744 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:10.861744 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:10.861744 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:10.861744 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:10.861744 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:10.861744 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:10.861744 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:10.861744 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:10.861744 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:10.861744 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:10.861744 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:10.861744 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:10.862526 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:10.862526 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:10.862526 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:10.862526 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:10.862526 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:35:10.862526 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:35:10.862526 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:35:10.862526 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:35:10.862526 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:35:10.862526 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:35:10.862526 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:35:10.862526 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:10.863582 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fe3546-18d0-44f1-bb39-446b023dffd7 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:10.863657 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 221/439] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 22:35:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:35:10.863657 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd2000d-71bb-4730-b53b-692eb583cf70 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:10.863777 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 219/440] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:35:10 2022] GET /identity/v3/projects/6edc20faecff41bf9b7ecc5ac4bc774e => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:35:10.863922 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd2000d-71bb-4730-b53b-692eb583cf70 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:10.863922 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd2000d-71bb-4730-b53b-692eb583cf70 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:10.865152 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 222/441] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:10.868251 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fe3546-18d0-44f1-bb39-446b023dffd7 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:10.871319 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd2000d-71bb-4730-b53b-692eb583cf70 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:10.937144 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:10.963209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:10.968986 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fe3546-18d0-44f1-bb39-446b023dffd7 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:10.970741 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd2000d-71bb-4730-b53b-692eb583cf70 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:11.003188 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eb8f96-f7d7-427e-97f2-8a912b2c3357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:11.003843 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb8f96-f7d7-427e-97f2-8a912b2c3357 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:11.004128 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb8f96-f7d7-427e-97f2-8a912b2c3357 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:11.004406 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb8f96-f7d7-427e-97f2-8a912b2c3357 None admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:11.004594 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d24d5-d686-4fd8-8687-f2860bdcccd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:11.005152 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eb8f96-f7d7-427e-97f2-8a912b2c3357 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:11.005431 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d24d5-d686-4fd8-8687-f2860bdcccd4 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:11.005530 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eb8f96-f7d7-427e-97f2-8a912b2c3357 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:11.005714 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d24d5-d686-4fd8-8687-f2860bdcccd4 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:11.005985 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d24d5-d686-4fd8-8687-f2860bdcccd4 None admin] PATH_INFO: `/v3/groups/5f723cf655a24fbfbf12b2dc2aa3e0c7` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:11.006836 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eb8f96-f7d7-427e-97f2-8a912b2c3357 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:11.007175 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6d24d5-d686-4fd8-8687-f2860bdcccd4 None admin] RBAC: Authorizing `identity:get_group(group_id=5f723cf655a24fbfbf12b2dc2aa3e0c7)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:11.008641 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6d24d5-d686-4fd8-8687-f2860bdcccd4 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:11.010243 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 223/442] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jul 21 22:35:10 2022] GET /identity/v3/groups/5f723cf655a24fbfbf12b2dc2aa3e0c7 => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:35:11.011375 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34739055-4ce1-40c1-b749-ae63f6942c85 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:11.011663 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34739055-4ce1-40c1-b749-ae63f6942c85 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:11.011941 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34739055-4ce1-40c1-b749-ae63f6942c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:11.019607 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 220/443] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:35:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:11.019607 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eb8f96-f7d7-427e-97f2-8a912b2c3357 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:11.030760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-34739055-4ce1-40c1-b749-ae63f6942c85 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:11.031977 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-34739055-4ce1-40c1-b749-ae63f6942c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:11.098035 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 224/444] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:35:11.098660 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34739055-4ce1-40c1-b749-ae63f6942c85 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:11.123389 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eb8f96-f7d7-427e-97f2-8a912b2c3357 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:11.157954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1741416e-c3bd-4656-a02c-4880321223dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:11.158607 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1741416e-c3bd-4656-a02c-4880321223dc None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:11.158928 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1741416e-c3bd-4656-a02c-4880321223dc None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:11.159205 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1741416e-c3bd-4656-a02c-4880321223dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:11.160034 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1741416e-c3bd-4656-a02c-4880321223dc None admin] RBAC: Adding query filter params (group.id=5f723cf655a24fbfbf12b2dc2aa3e0c7, role.id=53d3bdbe9cdf4d35bc73854a33df8e45, scope.project.id=6edc20faecff41bf9b7ecc5ac4bc774e) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:11.160349 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1741416e-c3bd-4656-a02c-4880321223dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:11.161779 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1741416e-c3bd-4656-a02c-4880321223dc None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:11.162949 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:11.167617 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 221/445] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Jul 21 22:35:11 2022] GET /identity/v3/role_assignments?group.id=5f723cf655a24fbfbf12b2dc2aa3e0c7&scope.project.id=6edc20faecff41bf9b7ecc5ac4bc774e&role.id=53d3bdbe9cdf4d35bc73854a33df8e45 => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:11.168397 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1741416e-c3bd-4656-a02c-4880321223dc None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:11.195307 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:11.207772 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34739055-4ce1-40c1-b749-ae63f6942c85 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:11.248181 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6c388e-a73f-4562-8b86-230783eb0764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:11.249274 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c388e-a73f-4562-8b86-230783eb0764 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:11.249770 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c388e-a73f-4562-8b86-230783eb0764 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:11.250108 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c388e-a73f-4562-8b86-230783eb0764 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:11.283860 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1741416e-c3bd-4656-a02c-4880321223dc None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:11.323049 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6182725-3911-4b53-b08d-62746ea309c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:11.323987 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6182725-3911-4b53-b08d-62746ea309c6 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:11.324276 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6182725-3911-4b53-b08d-62746ea309c6 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:11.324554 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6182725-3911-4b53-b08d-62746ea309c6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:11.326075 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6c388e-a73f-4562-8b86-230783eb0764 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:11.327567 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6c388e-a73f-4562-8b86-230783eb0764 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:11.340887 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6182725-3911-4b53-b08d-62746ea309c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:35:11.349181 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-f6182725-3911-4b53-b08d-62746ea309c6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:35:11.349181 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:11.349181 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:11.349181 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:11.349181 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:11.349181 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:11.349181 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:11.349181 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:11.349181 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:11.349181 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:11.349181 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:11.349181 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:11.349181 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:35:11.349181 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:35:11.349181 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:35:11.349181 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:35:11.350072 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:35:11.350072 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:35:11.350072 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:35:11.350072 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:35:11.350072 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:11.350072 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:11.350072 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:11.350072 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:11.350072 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:11.350072 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:11.350072 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:11.350072 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:11.350072 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:11.350072 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:11.350072 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:11.350072 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:11.350901 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:11.350901 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:11.350901 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:11.350901 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:11.350901 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:35:11.350901 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:35:11.350901 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:35:11.350901 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:35:11.350901 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:35:11.350901 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:35:11.350901 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:35:11.350901 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:11.350901 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 222/446] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 21 22:35:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:11.356878 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6182725-3911-4b53-b08d-62746ea309c6 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:11.408112 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:11.452042 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:11.455100 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 225/447] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:35:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:35:11.460783 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6c388e-a73f-4562-8b86-230783eb0764 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:11.470594 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6182725-3911-4b53-b08d-62746ea309c6 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:11.514018 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d412c-5b93-4d0d-b3e8-eeaef2f8f636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:11.515097 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d412c-5b93-4d0d-b3e8-eeaef2f8f636 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:11.515379 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d412c-5b93-4d0d-b3e8-eeaef2f8f636 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:11.515647 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d412c-5b93-4d0d-b3e8-eeaef2f8f636 None admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:11.516719 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79d412c-5b93-4d0d-b3e8-eeaef2f8f636 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:11.517024 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79d412c-5b93-4d0d-b3e8-eeaef2f8f636 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:11.518391 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79d412c-5b93-4d0d-b3e8-eeaef2f8f636 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:11.526943 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 223/448] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:35:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:35:11.533175 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d412c-5b93-4d0d-b3e8-eeaef2f8f636 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:11.594695 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6c388e-a73f-4562-8b86-230783eb0764 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:11.639746 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4121004-4222-44e8-85c4-aaf66538f98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:11.640535 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4121004-4222-44e8-85c4-aaf66538f98d None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:11.640812 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4121004-4222-44e8-85c4-aaf66538f98d None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:11.641085 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4121004-4222-44e8-85c4-aaf66538f98d None admin] PATH_INFO: `/v3/projects/d47c52ce443d40eaa4e635852fb964cd` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:11.643141 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4121004-4222-44e8-85c4-aaf66538f98d None admin] RBAC: Authorizing `identity:get_project(project_id=d47c52ce443d40eaa4e635852fb964cd)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:11.644519 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4121004-4222-44e8-85c4-aaf66538f98d None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:11.646237 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 226/449] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:35:11 2022] GET /identity/v3/projects/d47c52ce443d40eaa4e635852fb964cd => generated 344 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:35:11.651755 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4121004-4222-44e8-85c4-aaf66538f98d None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:11.652528 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d412c-5b93-4d0d-b3e8-eeaef2f8f636 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:11.690623 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:11.699123 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8335afc7-a99f-43c3-b05b-767aa4eba072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:11.705395 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8335afc7-a99f-43c3-b05b-767aa4eba072 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:11.705395 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8335afc7-a99f-43c3-b05b-767aa4eba072 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:11.705395 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8335afc7-a99f-43c3-b05b-767aa4eba072 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:11.718941 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:11.769829 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4121004-4222-44e8-85c4-aaf66538f98d None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:11.783306 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8335afc7-a99f-43c3-b05b-767aa4eba072 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:11.785117 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8335afc7-a99f-43c3-b05b-767aa4eba072 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:11.806390 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b435eb40-2834-4342-acba-93112b16a56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:11.807305 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435eb40-2834-4342-acba-93112b16a56e None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:11.807589 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435eb40-2834-4342-acba-93112b16a56e None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:11.807861 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435eb40-2834-4342-acba-93112b16a56e None admin] PATH_INFO: `/v3/groups/d77a5b66867941989c313e3a336978b3` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:11.809054 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b435eb40-2834-4342-acba-93112b16a56e None admin] RBAC: Authorizing `identity:get_group(group_id=d77a5b66867941989c313e3a336978b3)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:11.810478 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b435eb40-2834-4342-acba-93112b16a56e None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:11.812280 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 227/450] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jul 21 22:35:11 2022] GET /identity/v3/groups/d77a5b66867941989c313e3a336978b3 => generated 229 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:35:11.817839 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b435eb40-2834-4342-acba-93112b16a56e None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:11.892005 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 224/451] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:35:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:35:11.897408 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8335afc7-a99f-43c3-b05b-767aa4eba072 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:11.926903 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b435eb40-2834-4342-acba-93112b16a56e None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:11.928531 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:11.940646 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:11.959893 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae97e6-5d07-464a-8b12-61b6e506bd83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:11.960537 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae97e6-5d07-464a-8b12-61b6e506bd83 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:11.960824 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae97e6-5d07-464a-8b12-61b6e506bd83 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:11.961761 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae97e6-5d07-464a-8b12-61b6e506bd83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:11.962501 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ae97e6-5d07-464a-8b12-61b6e506bd83 None admin] RBAC: Adding query filter params (group.id=d77a5b66867941989c313e3a336978b3, role.id=d1ddee590a964ff1bbb4ee8716173021, scope.project.id=d47c52ce443d40eaa4e635852fb964cd) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:11.963109 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ae97e6-5d07-464a-8b12-61b6e506bd83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:11.965126 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ae97e6-5d07-464a-8b12-61b6e506bd83 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:11.970958 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 228/452] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Jul 21 22:35:11 2022] GET /identity/v3/role_assignments?group.id=d77a5b66867941989c313e3a336978b3&scope.project.id=d47c52ce443d40eaa4e635852fb964cd&role.id=d1ddee590a964ff1bbb4ee8716173021 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:35:11.994755 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8335afc7-a99f-43c3-b05b-767aa4eba072 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:12.030883 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11b43f5-1b7c-40b7-896c-04e054d43506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:12.031755 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b43f5-1b7c-40b7-896c-04e054d43506 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:12.032035 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b43f5-1b7c-40b7-896c-04e054d43506 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:12.032308 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b43f5-1b7c-40b7-896c-04e054d43506 None admin] PATH_INFO: `/v3/projects/a54cbea0067945899d2c8c75b7973660` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:12.033508 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11b43f5-1b7c-40b7-896c-04e054d43506 None admin] RBAC: Authorizing `identity:get_project(project_id=a54cbea0067945899d2c8c75b7973660)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:12.034984 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11b43f5-1b7c-40b7-896c-04e054d43506 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:12.036746 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 225/453] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:35:11 2022] GET /identity/v3/projects/a54cbea0067945899d2c8c75b7973660 => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:35:12.043074 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae97e6-5d07-464a-8b12-61b6e506bd83 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:12.148071 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae97e6-5d07-464a-8b12-61b6e506bd83 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:12.182674 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c795fa1c-fe3e-431c-9518-f80023fe761b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:12.183554 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795fa1c-fe3e-431c-9518-f80023fe761b None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:12.183838 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795fa1c-fe3e-431c-9518-f80023fe761b None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:12.184115 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795fa1c-fe3e-431c-9518-f80023fe761b None admin] PATH_INFO: `/v3/groups/5f723cf655a24fbfbf12b2dc2aa3e0c7` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:12.185281 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c795fa1c-fe3e-431c-9518-f80023fe761b None admin] RBAC: Authorizing `identity:get_group(group_id=5f723cf655a24fbfbf12b2dc2aa3e0c7)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:12.186712 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c795fa1c-fe3e-431c-9518-f80023fe761b None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:12.188361 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 229/454] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Jul 21 22:35:12 2022] GET /identity/v3/groups/5f723cf655a24fbfbf12b2dc2aa3e0c7 => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:35:12.194212 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11b43f5-1b7c-40b7-896c-04e054d43506 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:12.292137 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11b43f5-1b7c-40b7-896c-04e054d43506 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:12.304588 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:12.325189 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a512994a-51ab-441f-bfb8-08ac606b4f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:12.325836 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a512994a-51ab-441f-bfb8-08ac606b4f39 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:12.326118 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a512994a-51ab-441f-bfb8-08ac606b4f39 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:12.327529 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a512994a-51ab-441f-bfb8-08ac606b4f39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:12.327529 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a512994a-51ab-441f-bfb8-08ac606b4f39 None admin] RBAC: Adding query filter params (group.id=5f723cf655a24fbfbf12b2dc2aa3e0c7, role.id=b1849c34fc37410098b94ff687ab0055, scope.project.id=a54cbea0067945899d2c8c75b7973660) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:12.327683 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a512994a-51ab-441f-bfb8-08ac606b4f39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:12.329083 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a512994a-51ab-441f-bfb8-08ac606b4f39 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:12.335796 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 226/455] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Jul 21 22:35:12 2022] GET /identity/v3/role_assignments?group.id=5f723cf655a24fbfbf12b2dc2aa3e0c7&scope.project.id=a54cbea0067945899d2c8c75b7973660&role.id=b1849c34fc37410098b94ff687ab0055 => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:14.962801 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b981645c-9d46-4a00-9c64-69fe7cb3488c None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:14.963661 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b981645c-9d46-4a00-9c64-69fe7cb3488c None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:14.964113 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b981645c-9d46-4a00-9c64-69fe7cb3488c None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:14.965892 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 230/456] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:14.971621 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b34089-a659-4724-aa15-697c95592034 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:14.972364 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b34089-a659-4724-aa15-697c95592034 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:14.972787 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b34089-a659-4724-aa15-697c95592034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:14.990955 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-e2b34089-a659-4724-aa15-697c95592034 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:14.992426 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-e2b34089-a659-4724-aa15-697c95592034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:15.019424 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48708a96-1668-45ee-8699-75e6f28e5c6c None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:15.021756 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48708a96-1668-45ee-8699-75e6f28e5c6c None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:15.021756 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48708a96-1668-45ee-8699-75e6f28e5c6c None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:15.022288 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 231/457] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:15.027945 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8c708a-a1f0-4730-bb04-a686f309606b None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:15.028504 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8c708a-a1f0-4730-bb04-a686f309606b None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:15.028981 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8c708a-a1f0-4730-bb04-a686f309606b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:15.063746 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-cb8c708a-a1f0-4730-bb04-a686f309606b None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:15.065327 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-cb8c708a-a1f0-4730-bb04-a686f309606b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:15.066871 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 227/458] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:35:15.087625 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b34089-a659-4724-aa15-697c95592034 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:15.127816 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:15.190680 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b34089-a659-4724-aa15-697c95592034 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:15.202591 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:15.216172 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 232/459] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:35:15.227388 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79656eda-cd81-44b2-ac12-82711bcbe05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:15.228712 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79656eda-cd81-44b2-ac12-82711bcbe05e None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:15.229198 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79656eda-cd81-44b2-ac12-82711bcbe05e None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:15.229670 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79656eda-cd81-44b2-ac12-82711bcbe05e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:15.241063 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79656eda-cd81-44b2-ac12-82711bcbe05e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:15.243369 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79656eda-cd81-44b2-ac12-82711bcbe05e None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:15.250719 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8c708a-a1f0-4730-bb04-a686f309606b None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:15.262739 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-79656eda-cd81-44b2-ac12-82711bcbe05e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:15.262739 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:15.262739 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:15.262739 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:15.262739 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:15.262739 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:15.262739 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:15.262739 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:15.262739 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:15.262739 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:15.262739 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:15.262739 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:15.262739 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:15.262739 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:15.262739 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:15.263644 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:15.263644 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:15.263644 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:15.263644 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:15.263644 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:15.263644 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:15.263644 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:15.263644 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:15.263644 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:15.263644 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:15.263644 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:15.263644 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:15.263644 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:15.263644 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:15.263644 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:15.263644 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:15.263644 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:15.264515 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:15.264515 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:15.264515 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:15.264515 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:15.268813 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 228/460] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:15.270123 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79656eda-cd81-44b2-ac12-82711bcbe05e None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:15.379149 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79656eda-cd81-44b2-ac12-82711bcbe05e None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:15.432153 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893bff91-3520-432a-936d-6b61ca493a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:15.432818 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893bff91-3520-432a-936d-6b61ca493a7e None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:15.433095 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893bff91-3520-432a-936d-6b61ca493a7e None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:15.433365 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893bff91-3520-432a-936d-6b61ca493a7e None admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:15.434093 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893bff91-3520-432a-936d-6b61ca493a7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:15.434450 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893bff91-3520-432a-936d-6b61ca493a7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:15.435836 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893bff91-3520-432a-936d-6b61ca493a7e None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:15.458069 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 229/461] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:15.465198 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893bff91-3520-432a-936d-6b61ca493a7e None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:15.474752 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8c708a-a1f0-4730-bb04-a686f309606b None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:15.488116 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:15.535925 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913a1fa3-ba63-40bc-9b12-193c7af30508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:15.537036 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a1fa3-ba63-40bc-9b12-193c7af30508 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:15.537308 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a1fa3-ba63-40bc-9b12-193c7af30508 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:15.537752 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a1fa3-ba63-40bc-9b12-193c7af30508 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:15.548771 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913a1fa3-ba63-40bc-9b12-193c7af30508 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:15.550248 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913a1fa3-ba63-40bc-9b12-193c7af30508 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:15.570520 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-913a1fa3-ba63-40bc-9b12-193c7af30508 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:15.570520 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:15.570520 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:15.570520 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:15.570520 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:15.570520 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:15.570520 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:15.570520 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:15.570520 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:15.570520 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:15.570520 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:15.570520 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:15.570520 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:15.570520 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:15.570520 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:15.571433 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:15.571433 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:15.571433 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:15.571433 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:15.571433 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:15.571433 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:15.571433 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:15.571433 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:15.571433 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:15.571433 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:15.571433 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:15.571433 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:15.571433 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:15.571433 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:15.571433 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:15.571433 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:15.571433 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:15.572267 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:15.572267 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:15.572267 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:15.572267 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:15.572267 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 233/462] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:35:15.572983 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b78497-95db-4bb2-b1b6-4f2d654dd871 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:15.573268 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b78497-95db-4bb2-b1b6-4f2d654dd871 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:15.573542 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b78497-95db-4bb2-b1b6-4f2d654dd871 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:15.579357 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 234/463] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:15.580132 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b78497-95db-4bb2-b1b6-4f2d654dd871 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:15.585367 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893bff91-3520-432a-936d-6b61ca493a7e None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:15.608058 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:15.622378 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0aafc1-7efb-4eae-b607-b3c2f2fcc4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:15.623499 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0aafc1-7efb-4eae-b607-b3c2f2fcc4e1 None admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:15.623788 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0aafc1-7efb-4eae-b607-b3c2f2fcc4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:15.624153 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0aafc1-7efb-4eae-b607-b3c2f2fcc4e1 None admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:15.624861 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0aafc1-7efb-4eae-b607-b3c2f2fcc4e1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:15.626304 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0aafc1-7efb-4eae-b607-b3c2f2fcc4e1 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:15.695482 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b78497-95db-4bb2-b1b6-4f2d654dd871 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:15.715238 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 230/464] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 21 22:35:15 2022] POST /identity/v3/users => generated 248 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:15.716876 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194dea1-cfb1-47a9-8722-018d6f0f5bff None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:15.717169 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194dea1-cfb1-47a9-8722-018d6f0f5bff None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:15.717457 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194dea1-cfb1-47a9-8722-018d6f0f5bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:15.735819 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413fa551-1da5-4403-9708-49e9c90a82de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:15.736435 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413fa551-1da5-4403-9708-49e9c90a82de None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:15.736721 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413fa551-1da5-4403-9708-49e9c90a82de None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:15.736999 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413fa551-1da5-4403-9708-49e9c90a82de None admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:15.737757 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413fa551-1da5-4403-9708-49e9c90a82de None admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:15.738066 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413fa551-1da5-4403-9708-49e9c90a82de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:15.740459 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413fa551-1da5-4403-9708-49e9c90a82de None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:15.746736 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-e194dea1-cfb1-47a9-8722-018d6f0f5bff None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:15.748472 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-e194dea1-cfb1-47a9-8722-018d6f0f5bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:15.775004 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 235/465] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:15.776897 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918c4c6-7d3c-4a1e-842d-b2039b1af114 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:15.777429 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918c4c6-7d3c-4a1e-842d-b2039b1af114 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:15.779639 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918c4c6-7d3c-4a1e-842d-b2039b1af114 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:15.781682 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 236/466] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:15.783103 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0918c4c6-7d3c-4a1e-842d-b2039b1af114 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:15.846438 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 231/467] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:35:15.847969 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0ef5a-b4d6-4ac2-9898-616d601b131b None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:15.848257 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0ef5a-b4d6-4ac2-9898-616d601b131b None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:15.848573 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0ef5a-b4d6-4ac2-9898-616d601b131b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:15.857825 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:15.869378 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-7cf0ef5a-b4d6-4ac2-9898-616d601b131b None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:15.870947 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-7cf0ef5a-b4d6-4ac2-9898-616d601b131b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:15.883682 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:15.921061 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0918c4c6-7d3c-4a1e-842d-b2039b1af114 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:15.963229 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 232/468] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:35:15.966821 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf0ef5a-b4d6-4ac2-9898-616d601b131b None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:15.971495 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d295a650-2db3-4d5b-87c3-f26ef0359e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:15.972178 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d295a650-2db3-4d5b-87c3-f26ef0359e67 None admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:15.972463 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d295a650-2db3-4d5b-87c3-f26ef0359e67 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:15.972738 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d295a650-2db3-4d5b-87c3-f26ef0359e67 None admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:15.973454 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d295a650-2db3-4d5b-87c3-f26ef0359e67 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:15.974930 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d295a650-2db3-4d5b-87c3-f26ef0359e67 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:16.056695 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 237/469] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 21 22:35:15 2022] POST /identity/v3/users => generated 246 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:16.057452 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d295a650-2db3-4d5b-87c3-f26ef0359e67 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:16.109274 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf0ef5a-b4d6-4ac2-9898-616d601b131b None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:16.180685 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a57995-6b99-41c7-b02e-18c1d96c3e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:16.181513 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a57995-6b99-41c7-b02e-18c1d96c3e07 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:16.181793 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a57995-6b99-41c7-b02e-18c1d96c3e07 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:16.182070 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a57995-6b99-41c7-b02e-18c1d96c3e07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:16.184937 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:16.206494 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a57995-6b99-41c7-b02e-18c1d96c3e07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:16.208011 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a57995-6b99-41c7-b02e-18c1d96c3e07 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:16.248509 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-27a57995-6b99-41c7-b02e-18c1d96c3e07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:16.248509 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:16.248509 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:16.248509 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:16.248509 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:16.248509 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:16.248509 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:16.248509 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:16.248509 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:16.248509 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:16.248509 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:16.248509 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:16.248509 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:16.248509 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:16.248509 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:16.249440 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:16.249440 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:16.249440 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:16.249440 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:16.249440 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:16.249440 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:16.249440 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:16.249440 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:16.249440 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:16.249440 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:16.249440 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:16.249440 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:16.249440 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:16.249440 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:16.249440 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:16.249440 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:16.249440 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:16.250395 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:16.250395 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:16.250395 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:16.250395 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:16.253950 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 233/470] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:16.257087 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a57995-6b99-41c7-b02e-18c1d96c3e07 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:16.265920 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d295a650-2db3-4d5b-87c3-f26ef0359e67 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:16.297627 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:16.303196 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a0e3f1-c223-4e1d-ab7b-c747c0d62026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:16.304412 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a0e3f1-c223-4e1d-ab7b-c747c0d62026 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:16.304412 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a0e3f1-c223-4e1d-ab7b-c747c0d62026 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:16.304642 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a0e3f1-c223-4e1d-ab7b-c747c0d62026 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:16.317529 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a0e3f1-c223-4e1d-ab7b-c747c0d62026 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:16.318028 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a0e3f1-c223-4e1d-ab7b-c747c0d62026 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:16.339829 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-02a0e3f1-c223-4e1d-ab7b-c747c0d62026 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:16.339829 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:16.339829 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:16.339829 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:16.339829 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:16.339829 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:16.339829 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:16.339829 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:16.339829 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:16.339829 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:16.339829 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:16.339829 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:16.339829 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:16.339829 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:16.339829 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:16.340746 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:16.340746 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:16.340746 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:16.340746 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:16.340746 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:16.340746 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:16.340746 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:16.340746 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:16.340746 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:16.340746 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:16.340746 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:16.340746 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:16.340746 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:16.340746 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:16.340746 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:16.340746 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:16.340746 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:16.341914 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:16.341914 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:16.341914 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:16.341914 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:16.341914 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 238/471] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:16.347150 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a0e3f1-c223-4e1d-ab7b-c747c0d62026 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:16.381489 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a57995-6b99-41c7-b02e-18c1d96c3e07 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:16.427973 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca033564-b372-4aa8-ba07-e1304a638efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:16.429425 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca033564-b372-4aa8-ba07-e1304a638efe None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:16.430386 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca033564-b372-4aa8-ba07-e1304a638efe None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:16.430833 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca033564-b372-4aa8-ba07-e1304a638efe None admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:16.432003 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca033564-b372-4aa8-ba07-e1304a638efe None admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:16.432458 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca033564-b372-4aa8-ba07-e1304a638efe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:16.434902 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca033564-b372-4aa8-ba07-e1304a638efe None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:16.454340 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 234/472] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:16.460097 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca033564-b372-4aa8-ba07-e1304a638efe None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:16.484649 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a0e3f1-c223-4e1d-ab7b-c747c0d62026 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:16.518082 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:16.533556 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5236f3ad-8fac-42b2-a016-6cbfc2ac8460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:16.534377 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5236f3ad-8fac-42b2-a016-6cbfc2ac8460 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:16.535004 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5236f3ad-8fac-42b2-a016-6cbfc2ac8460 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:16.535288 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5236f3ad-8fac-42b2-a016-6cbfc2ac8460 None admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:16.536295 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5236f3ad-8fac-42b2-a016-6cbfc2ac8460 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:16.536607 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5236f3ad-8fac-42b2-a016-6cbfc2ac8460 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:16.537998 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5236f3ad-8fac-42b2-a016-6cbfc2ac8460 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:16.555272 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 239/473] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:16.557377 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd600448-b82c-4a52-b8ea-a2623efe35d9 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:16.559116 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd600448-b82c-4a52-b8ea-a2623efe35d9 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:16.567904 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd600448-b82c-4a52-b8ea-a2623efe35d9 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:16.567904 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 240/474] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:16 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:16.567904 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd600448-b82c-4a52-b8ea-a2623efe35d9 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:16.581582 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca033564-b372-4aa8-ba07-e1304a638efe None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:16.621618 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9f2c1a-6807-4914-9724-2842a88bcd76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:16.622842 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f2c1a-6807-4914-9724-2842a88bcd76 None admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:16.623331 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f2c1a-6807-4914-9724-2842a88bcd76 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:16.623933 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f2c1a-6807-4914-9724-2842a88bcd76 None admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:16.624837 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9f2c1a-6807-4914-9724-2842a88bcd76 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:16.626583 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9f2c1a-6807-4914-9724-2842a88bcd76 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:16.628041 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:16.687373 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd600448-b82c-4a52-b8ea-a2623efe35d9 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:16.723375 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dbdead-5434-4756-a515-742050b9ad34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:16.724036 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dbdead-5434-4756-a515-742050b9ad34 None admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:16.724323 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dbdead-5434-4756-a515-742050b9ad34 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:16.724636 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dbdead-5434-4756-a515-742050b9ad34 None admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:16.725305 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dbdead-5434-4756-a515-742050b9ad34 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:16.727358 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dbdead-5434-4756-a515-742050b9ad34 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:16.747939 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 235/475] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 21 22:35:16 2022] POST /identity/v3/users => generated 248 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:16.749719 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e9fc9-61c8-41f4-8b5e-74d4469b8779 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:16.750218 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e9fc9-61c8-41f4-8b5e-74d4469b8779 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:16.750677 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e9fc9-61c8-41f4-8b5e-74d4469b8779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:16.774216 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-502e9fc9-61c8-41f4-8b5e-74d4469b8779 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:16.775949 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-502e9fc9-61c8-41f4-8b5e-74d4469b8779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:16.824636 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 241/476] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 21 22:35:16 2022] POST /identity/v3/users => generated 249 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:16.869966 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 236/477] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:35:16.906042 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dbdead-5434-4756-a515-742050b9ad34 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:17.032781 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:17.036739 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dbdead-5434-4756-a515-742050b9ad34 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:17.074219 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c32f65d-83aa-439b-ae7e-1e17314570d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:17.075030 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c32f65d-83aa-439b-ae7e-1e17314570d4 None admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:17.075331 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c32f65d-83aa-439b-ae7e-1e17314570d4 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:17.075680 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c32f65d-83aa-439b-ae7e-1e17314570d4 None admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:17.076440 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c32f65d-83aa-439b-ae7e-1e17314570d4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:17.077843 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c32f65d-83aa-439b-ae7e-1e17314570d4 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:17.129990 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.sql.core [None req-3c32f65d-83aa-439b-ae7e-1e17314570d4 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 21 22:35:17.129990 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 21 22:35:17.129990 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [parameters: {'id': 'c3aacc21f9aa4224bfacb41607a5a463', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 21 22:35:17.129990 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87167) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 21 22:35:17.147152 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-3c32f65d-83aa-439b-ae7e-1e17314570d4 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 21 22:35:17.147152 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:17.147152 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 22:35:17.147152 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 22:35:17.147152 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 22:35:17.147152 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 22:35:17.147152 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 22:35:17.147152 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 22:35:17.147152 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 22:35:17.147152 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application conn.query(q) Jul 21 22:35:17.147152 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 22:35:17.147152 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 22:35:17.147152 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 22:35:17.147152 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application result.read() Jul 21 22:35:17.147152 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 22:35:17.148040 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 22:35:17.148040 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 22:35:17.148040 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 22:35:17.148040 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 22:35:17.148040 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 22:35:17.148040 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 22:35:17.148040 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 22:35:17.148040 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 21 22:35:17.148040 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:17.148040 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 21 22:35:17.148040 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:17.148040 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:17.148040 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 21 22:35:17.148040 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 21 22:35:17.148040 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 21 22:35:17.148040 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return ref.to_dict() Jul 21 22:35:17.148040 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 22:35:17.148040 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application next(self.gen) Jul 21 22:35:17.149990 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 21 22:35:17.149990 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application yield resource Jul 21 22:35:17.149990 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 22:35:17.149990 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application next(self.gen) Jul 21 22:35:17.149990 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 21 22:35:17.149990 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self.session.rollback() Jul 21 22:35:17.149990 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 21 22:35:17.149990 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self.force_reraise() Jul 21 22:35:17.149990 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 21 22:35:17.149990 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise self.value Jul 21 22:35:17.149990 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 21 22:35:17.149990 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 21 22:35:17.149990 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 21 22:35:17.149990 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application session.commit() Jul 21 22:35:17.149990 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 21 22:35:17.149990 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 21 22:35:17.149990 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 21 22:35:17.151103 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self._prepare_impl() Jul 21 22:35:17.151103 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 21 22:35:17.151103 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self.session.flush() Jul 21 22:35:17.151103 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 21 22:35:17.151103 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self._flush(objects) Jul 21 22:35:17.151103 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 21 22:35:17.151103 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 21 22:35:17.151103 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 21 22:35:17.151103 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application compat.raise_( Jul 21 22:35:17.151103 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 22:35:17.151103 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception Jul 21 22:35:17.151103 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 21 22:35:17.151103 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application flush_context.execute() Jul 21 22:35:17.151103 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 21 22:35:17.151103 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rec.execute(self) Jul 21 22:35:17.151103 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 21 22:35:17.151103 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 21 22:35:17.151961 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 21 22:35:17.151961 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 21 22:35:17.151961 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 21 22:35:17.151961 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 21 22:35:17.151961 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 21 22:35:17.151961 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 21 22:35:17.151961 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 21 22:35:17.151961 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 21 22:35:17.151961 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 21 22:35:17.151961 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 21 22:35:17.151961 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 21 22:35:17.151961 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 21 22:35:17.151961 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 21 22:35:17.151961 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 21 22:35:17.151961 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 22:35:17.151961 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception Jul 21 22:35:17.152760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 22:35:17.152760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 22:35:17.152760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 22:35:17.152760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 22:35:17.152760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 22:35:17.152760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 22:35:17.152760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 22:35:17.152760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application conn.query(q) Jul 21 22:35:17.152760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 22:35:17.152760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 22:35:17.152760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 22:35:17.152760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application result.read() Jul 21 22:35:17.152760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 22:35:17.152760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 22:35:17.152760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 22:35:17.152760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 22:35:17.154392 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 22:35:17.154392 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 22:35:17.154392 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 22:35:17.154392 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 22:35:17.154392 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 21 22:35:17.154392 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 21 22:35:17.154392 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [parameters: {'id': 'c3aacc21f9aa4224bfacb41607a5a463', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 21 22:35:17.154392 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 21 22:35:17.154392 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:17.154392 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:35:17.154392 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:17.154392 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:17.154392 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:17.154392 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:17.154392 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:17.154392 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:17.156485 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:17.156485 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:17.156485 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:17.156485 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:17.156485 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:17.156485 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:17.156485 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 21 22:35:17.156485 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 21 22:35:17.156485 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:17.156485 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:17.156485 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 21 22:35:17.156485 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 21 22:35:17.156485 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 21 22:35:17.156485 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 21 22:35:17.156485 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 21 22:35:17.156485 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:17.167124 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 242/478] 10.222.0.52 () {60 vars in 1131 bytes} [Thu Jul 21 22:35:16 2022] POST /identity/v3/roles => generated 144 bytes in 244 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:17.167269 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e9fc9-61c8-41f4-8b5e-74d4469b8779 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:17.203960 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:17.278690 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e9fc9-61c8-41f4-8b5e-74d4469b8779 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:17.320209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d7b064-bd67-4ca3-b355-883c5af63a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:17.321322 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d7b064-bd67-4ca3-b355-883c5af63a66 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:17.321832 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d7b064-bd67-4ca3-b355-883c5af63a66 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:17.322252 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d7b064-bd67-4ca3-b355-883c5af63a66 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:17.337778 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d7b064-bd67-4ca3-b355-883c5af63a66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:35:17.346030 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-59d7b064-bd67-4ca3-b355-883c5af63a66 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:35:17.346030 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:17.346030 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:17.346030 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:17.346030 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:17.346030 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:17.346030 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:17.346030 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:17.346030 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:17.346030 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:17.346030 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:17.346030 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:17.346030 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:35:17.346030 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:35:17.346030 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:35:17.346030 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:35:17.349635 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:35:17.349635 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:35:17.349635 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:35:17.349635 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:35:17.349635 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:17.349635 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:17.349635 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:17.349635 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:17.349635 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:17.349635 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:17.349635 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:17.349635 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:17.349635 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:17.349635 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:17.349635 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:17.349635 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:17.350636 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:17.350636 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:17.350636 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:17.350636 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:17.350636 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:35:17.350636 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:35:17.350636 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:35:17.350636 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:35:17.350636 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:35:17.350636 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:35:17.350636 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 22:35:17.350636 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:17.354588 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 237/479] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 21 22:35:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:17.362596 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c32f65d-83aa-439b-ae7e-1e17314570d4 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:17.495313 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c32f65d-83aa-439b-ae7e-1e17314570d4 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:17.555382 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02907fc5-e483-4d81-bf31-2b7ec9c135e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:17.556118 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02907fc5-e483-4d81-bf31-2b7ec9c135e1 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:17.556409 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02907fc5-e483-4d81-bf31-2b7ec9c135e1 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:17.556688 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02907fc5-e483-4d81-bf31-2b7ec9c135e1 None admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:17.557458 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02907fc5-e483-4d81-bf31-2b7ec9c135e1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:17.557770 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02907fc5-e483-4d81-bf31-2b7ec9c135e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:17.559176 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02907fc5-e483-4d81-bf31-2b7ec9c135e1 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:17.570980 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 243/480] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:35:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:18.022707 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48deee-96d1-4995-bbeb-48a5a3b4f6cc None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:18.023601 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48deee-96d1-4995-bbeb-48a5a3b4f6cc None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:18.024000 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48deee-96d1-4995-bbeb-48a5a3b4f6cc None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:18.025587 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 238/481] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:35:18.031090 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48836a19-12c8-42ac-81e1-d3018f852d7e None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:18.031388 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48836a19-12c8-42ac-81e1-d3018f852d7e None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:18.031672 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48836a19-12c8-42ac-81e1-d3018f852d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:18.051552 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-48836a19-12c8-42ac-81e1-d3018f852d7e None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:18.052756 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-48836a19-12c8-42ac-81e1-d3018f852d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:18.076399 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:18.130299 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 244/482] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:35:18.153762 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e48deee-96d1-4995-bbeb-48a5a3b4f6cc None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:18.296236 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:18.306135 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e48deee-96d1-4995-bbeb-48a5a3b4f6cc None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:18.360372 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a00f8b-aa4e-43a9-92ad-9dc782c1bf8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:18.361172 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a00f8b-aa4e-43a9-92ad-9dc782c1bf8d None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:18.361454 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a00f8b-aa4e-43a9-92ad-9dc782c1bf8d None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:18.361728 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a00f8b-aa4e-43a9-92ad-9dc782c1bf8d None admin] PATH_INFO: `/v3/roles/service` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:18.376862 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a00f8b-aa4e-43a9-92ad-9dc782c1bf8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:35:18.386221 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-17a00f8b-aa4e-43a9-92ad-9dc782c1bf8d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:35:18.386221 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:18.386221 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:18.386221 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:18.386221 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:18.386221 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:18.386221 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:18.386221 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:18.386221 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:18.386221 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:18.386221 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:18.386221 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:18.386221 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:35:18.386221 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:35:18.386221 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:35:18.386221 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:35:18.387141 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:35:18.387141 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:35:18.387141 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:35:18.387141 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:35:18.387141 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:18.387141 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:18.387141 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:18.387141 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:18.387141 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:18.387141 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:18.387141 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:18.387141 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:18.387141 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:18.387141 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:18.387141 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:18.387141 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:18.387934 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:18.387934 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:18.387934 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:18.387934 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:18.387934 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:35:18.387934 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:35:18.387934 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:35:18.387934 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:35:18.387934 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:35:18.387934 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:35:18.387934 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:35:18.387934 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:18.387934 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 239/483] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 21 22:35:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:18.393438 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48836a19-12c8-42ac-81e1-d3018f852d7e None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:18.558741 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48836a19-12c8-42ac-81e1-d3018f852d7e None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:18.609898 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0412298-9922-4f45-91bc-af76cc75282e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:18.610570 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0412298-9922-4f45-91bc-af76cc75282e None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:18.610884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0412298-9922-4f45-91bc-af76cc75282e None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:18.611158 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0412298-9922-4f45-91bc-af76cc75282e None admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:18.611915 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0412298-9922-4f45-91bc-af76cc75282e None admin] RBAC: Adding query filter params (name=service) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:18.612218 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0412298-9922-4f45-91bc-af76cc75282e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:18.613594 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0412298-9922-4f45-91bc-af76cc75282e None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:18.626048 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 245/484] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 21 22:35:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:18.632118 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a00f8b-aa4e-43a9-92ad-9dc782c1bf8d None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:18.807412 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a00f8b-aa4e-43a9-92ad-9dc782c1bf8d None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:18.857489 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9bbca1-3282-4fef-82a7-6b045df7bb83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:18.858287 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9bbca1-3282-4fef-82a7-6b045df7bb83 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:18.858568 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9bbca1-3282-4fef-82a7-6b045df7bb83 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:18.858866 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9bbca1-3282-4fef-82a7-6b045df7bb83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:18.878929 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9bbca1-3282-4fef-82a7-6b045df7bb83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:18.880396 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9bbca1-3282-4fef-82a7-6b045df7bb83 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:18.901570 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-6b9bbca1-3282-4fef-82a7-6b045df7bb83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:18.901570 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:18.901570 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:18.901570 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:18.901570 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:18.901570 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:18.901570 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:18.901570 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:18.901570 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:18.901570 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:18.901570 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:18.901570 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:18.901570 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:18.901570 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:18.901570 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:18.902464 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:18.902464 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:18.902464 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:18.902464 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:18.902464 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:18.902464 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:18.902464 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:18.902464 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:18.902464 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:18.902464 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:18.902464 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:18.902464 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:18.902464 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:18.902464 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:18.902464 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:18.902464 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:18.902464 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:18.903328 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:18.903328 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:18.903328 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:18.903328 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:18.903328 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 240/485] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:18.908591 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0412298-9922-4f45-91bc-af76cc75282e None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:18.933746 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:19.047485 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0412298-9922-4f45-91bc-af76cc75282e None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:19.098323 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60237fbc-89d5-41ff-9624-9c66b1c2f314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:19.099007 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60237fbc-89d5-41ff-9624-9c66b1c2f314 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:19.099294 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60237fbc-89d5-41ff-9624-9c66b1c2f314 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:19.099574 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60237fbc-89d5-41ff-9624-9c66b1c2f314 None admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:19.100310 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60237fbc-89d5-41ff-9624-9c66b1c2f314 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:19.100618 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60237fbc-89d5-41ff-9624-9c66b1c2f314 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:19.102028 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60237fbc-89d5-41ff-9624-9c66b1c2f314 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:19.117560 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 246/486] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:19.123077 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9bbca1-3282-4fef-82a7-6b045df7bb83 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:19.127021 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:19.140165 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ff9f6-8432-41f9-bc72-86e8b74455df None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:19.140455 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ff9f6-8432-41f9-bc72-86e8b74455df None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:19.144539 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ff9f6-8432-41f9-bc72-86e8b74455df None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:19.144539 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 247/487] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:35:19.150831 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b30e5-df17-4def-b9fd-ab607424e354 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:19.151120 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b30e5-df17-4def-b9fd-ab607424e354 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:19.151432 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b30e5-df17-4def-b9fd-ab607424e354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:19.171794 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-7e8b30e5-df17-4def-b9fd-ab607424e354 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:19.173094 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-7e8b30e5-df17-4def-b9fd-ab607424e354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:19.243694 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 248/488] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:35:19.245560 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b632f475-0e65-43b1-9f75-73aa16a61ab4 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:19.245994 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b632f475-0e65-43b1-9f75-73aa16a61ab4 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:19.246415 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b632f475-0e65-43b1-9f75-73aa16a61ab4 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:19.247891 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 249/489] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:19.256067 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf81d0e7-b954-4607-8cd0-9b80d01cfe60 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:19.256327 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf81d0e7-b954-4607-8cd0-9b80d01cfe60 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:19.258791 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf81d0e7-b954-4607-8cd0-9b80d01cfe60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:19.275704 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-bf81d0e7-b954-4607-8cd0-9b80d01cfe60 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:19.276895 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-bf81d0e7-b954-4607-8cd0-9b80d01cfe60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:19.279076 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:19.292727 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9bbca1-3282-4fef-82a7-6b045df7bb83 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:19.340601 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ecf1f0-ffb0-41c3-ba1a-46ddba63dc7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:19.341499 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ecf1f0-ffb0-41c3-ba1a-46ddba63dc7b None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:19.341928 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ecf1f0-ffb0-41c3-ba1a-46ddba63dc7b None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:19.342214 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ecf1f0-ffb0-41c3-ba1a-46ddba63dc7b None admin] PATH_INFO: `/v3/users/glance` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:19.350920 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ecf1f0-ffb0-41c3-ba1a-46ddba63dc7b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:19.353145 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ecf1f0-ffb0-41c3-ba1a-46ddba63dc7b None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:19.358709 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 250/490] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:35:19.359708 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf81d0e7-b954-4607-8cd0-9b80d01cfe60 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:19.369891 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-30ecf1f0-ffb0-41c3-ba1a-46ddba63dc7b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 21 22:35:19.369891 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:19.369891 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:19.369891 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:19.369891 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:19.369891 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:19.369891 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:19.369891 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:19.369891 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:19.369891 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:19.369891 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:19.369891 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:19.369891 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:35:19.369891 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:35:19.369891 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:35:19.369891 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:35:19.370923 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:19.370923 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:19.370923 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:35:19.370923 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:35:19.370923 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:35:19.370923 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:35:19.370923 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:19.370923 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:19.370923 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:19.370923 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:19.370923 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:19.370923 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:19.370923 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:19.370923 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:19.370923 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:19.370923 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:19.370923 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:19.370923 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:19.372071 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:19.372071 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:19.372071 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:35:19.372071 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:35:19.372071 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:35:19.372071 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:35:19.372071 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:35:19.372071 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:35:19.372071 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 21 22:35:19.372071 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:19.372071 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 241/491] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:35:19 2022] GET /identity/v3/users/glance => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:19.378751 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ecf1f0-ffb0-41c3-ba1a-46ddba63dc7b None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:19.474323 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:19.485266 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf81d0e7-b954-4607-8cd0-9b80d01cfe60 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:19.494307 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ecf1f0-ffb0-41c3-ba1a-46ddba63dc7b None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:19.525373 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0f483b-1616-467a-beaa-fdc98e414fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:19.526254 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0f483b-1616-467a-beaa-fdc98e414fc8 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:19.526441 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0f483b-1616-467a-beaa-fdc98e414fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:19.526716 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0f483b-1616-467a-beaa-fdc98e414fc8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:19.528551 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3b06aa-d918-4079-b089-b107014cb67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:19.529228 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3b06aa-d918-4079-b089-b107014cb67a None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:19.529512 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3b06aa-d918-4079-b089-b107014cb67a None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:19.529791 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3b06aa-d918-4079-b089-b107014cb67a None admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:19.530637 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3b06aa-d918-4079-b089-b107014cb67a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:19.531059 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3b06aa-d918-4079-b089-b107014cb67a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:19.532500 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3b06aa-d918-4079-b089-b107014cb67a None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:19.535205 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:19.539674 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0f483b-1616-467a-beaa-fdc98e414fc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:35:19.548414 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-cb0f483b-1616-467a-beaa-fdc98e414fc8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:35:19.548414 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:19.548414 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:19.548414 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:19.548414 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:19.548414 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:19.548414 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:19.548414 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:19.548414 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:19.548414 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:19.548414 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:19.548414 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:19.548414 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:35:19.548414 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:35:19.548414 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:35:19.548414 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:35:19.551819 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:35:19.551819 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:35:19.551819 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:35:19.551819 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:35:19.551819 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:19.551819 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:19.551819 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:19.551819 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:19.551819 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:19.551819 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:19.551819 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:19.551819 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:19.551819 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:19.551819 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:19.551819 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:19.551819 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:19.552649 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:19.552649 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:19.552649 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:19.552649 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:19.552649 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:35:19.552649 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:35:19.552649 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:35:19.552649 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:35:19.552649 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:35:19.552649 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:35:19.552649 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:35:19.552649 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:19.552649 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 251/492] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 21 22:35:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:19.552649 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0f483b-1616-467a-beaa-fdc98e414fc8 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:19.578562 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 242/493] 10.222.0.52 () {56 vars in 1137 bytes} [Thu Jul 21 22:35:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:19.578562 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd62bf77-8649-4192-8215-17230b4409e1 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:19.578562 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd62bf77-8649-4192-8215-17230b4409e1 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:19.578562 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd62bf77-8649-4192-8215-17230b4409e1 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:19.578562 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 243/494] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:19.578562 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd62bf77-8649-4192-8215-17230b4409e1 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:19.667368 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0f483b-1616-467a-beaa-fdc98e414fc8 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:19.689915 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd62bf77-8649-4192-8215-17230b4409e1 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:19.701408 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2945ba11-f881-46ce-9883-aa9792387f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:19.702193 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2945ba11-f881-46ce-9883-aa9792387f18 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:19.702481 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2945ba11-f881-46ce-9883-aa9792387f18 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:19.702752 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2945ba11-f881-46ce-9883-aa9792387f18 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:19.714487 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:19.714863 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2945ba11-f881-46ce-9883-aa9792387f18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:35:19.725325 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-2945ba11-f881-46ce-9883-aa9792387f18 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:35:19.725325 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:19.725325 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:19.725325 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:19.725325 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:19.725325 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:19.725325 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:19.725325 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:19.725325 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:19.725325 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:19.725325 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:19.725325 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:19.725325 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:35:19.725325 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:35:19.725325 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:35:19.725325 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:35:19.727245 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:35:19.727245 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:35:19.727245 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:35:19.727245 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:35:19.727245 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:19.727245 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:19.727245 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:19.727245 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:19.727245 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:19.727245 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:19.727245 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:19.727245 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:19.727245 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:19.727245 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:19.727245 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:19.727245 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:19.728299 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:19.728299 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:19.728299 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:19.728299 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:19.728299 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:35:19.728299 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:35:19.728299 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:35:19.728299 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:35:19.728299 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:35:19.728299 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:35:19.728299 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:35:19.728299 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:19.728893 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c88292-bc6a-4c02-b348-31ea5d1bcf7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:19.728893 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c88292-bc6a-4c02-b348-31ea5d1bcf7f None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:19.729386 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c88292-bc6a-4c02-b348-31ea5d1bcf7f None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:19.729477 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c88292-bc6a-4c02-b348-31ea5d1bcf7f None admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:19.730209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c88292-bc6a-4c02-b348-31ea5d1bcf7f None admin] RBAC: Adding query filter params (name=service) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:19.730521 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c88292-bc6a-4c02-b348-31ea5d1bcf7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:19.732170 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c88292-bc6a-4c02-b348-31ea5d1bcf7f None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:19.732328 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 252/495] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 21 22:35:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:19.732913 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2945ba11-f881-46ce-9883-aa9792387f18 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:19.741958 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 244/496] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 21 22:35:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:19.743368 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af3950-455b-42b0-8b22-4909b9417708 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:19.743689 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af3950-455b-42b0-8b22-4909b9417708 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:19.743983 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af3950-455b-42b0-8b22-4909b9417708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:19.762811 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-71af3950-455b-42b0-8b22-4909b9417708 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:19.764013 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-71af3950-455b-42b0-8b22-4909b9417708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:19.798247 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:19.835449 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 245/497] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:35:19.842954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af3950-455b-42b0-8b22-4909b9417708 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:19.850254 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2945ba11-f881-46ce-9883-aa9792387f18 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:19.888082 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6a5cef-0e1e-4e62-8e52-7d8c36ce0eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:19.888850 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a5cef-0e1e-4e62-8e52-7d8c36ce0eb2 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:19.889209 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a5cef-0e1e-4e62-8e52-7d8c36ce0eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:19.889558 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a5cef-0e1e-4e62-8e52-7d8c36ce0eb2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:19.952447 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af3950-455b-42b0-8b22-4909b9417708 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:19.965299 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6a5cef-0e1e-4e62-8e52-7d8c36ce0eb2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:19.966980 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6a5cef-0e1e-4e62-8e52-7d8c36ce0eb2 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:19.967463 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:19.987856 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d986b6f-c4c4-4322-bf08-d8e89a870092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:19.988824 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d986b6f-c4c4-4322-bf08-d8e89a870092 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:19.989110 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d986b6f-c4c4-4322-bf08-d8e89a870092 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:19.989381 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d986b6f-c4c4-4322-bf08-d8e89a870092 None admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:19.990137 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d986b6f-c4c4-4322-bf08-d8e89a870092 None admin] RBAC: Adding query filter params (name=service) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:19.990445 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d986b6f-c4c4-4322-bf08-d8e89a870092 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:19.992346 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d986b6f-c4c4-4322-bf08-d8e89a870092 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:20.002485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 246/498] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 21 22:35:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:20.003680 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d986b6f-c4c4-4322-bf08-d8e89a870092 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:20.035187 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:20.078640 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 253/499] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:35:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:35:20.079833 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6a5cef-0e1e-4e62-8e52-7d8c36ce0eb2 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:20.116698 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d986b6f-c4c4-4322-bf08-d8e89a870092 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:20.155668 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ef8695-e39e-4e62-8705-987544f935d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:20.156494 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef8695-e39e-4e62-8705-987544f935d6 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:20.156778 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef8695-e39e-4e62-8705-987544f935d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:20.157055 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef8695-e39e-4e62-8705-987544f935d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:20.169211 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ef8695-e39e-4e62-8705-987544f935d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:20.170922 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ef8695-e39e-4e62-8705-987544f935d6 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:20.189069 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-87ef8695-e39e-4e62-8705-987544f935d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:20.189069 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:20.189069 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:20.189069 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:20.189069 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:20.189069 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:20.189069 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:20.189069 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:20.189069 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:20.189069 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:20.189069 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:20.189069 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:20.189069 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:20.189069 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:20.189069 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:20.189931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:20.189931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:20.189931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:20.189931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:20.189931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:20.189931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:20.189931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:20.189931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:20.189931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:20.189931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:20.189931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:20.189931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:20.189931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:20.189931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:20.189931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:20.189931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:20.189931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:20.196230 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:20.196230 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:20.196230 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:20.196230 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:20.196230 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 247/500] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:20.196493 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6a5cef-0e1e-4e62-8e52-7d8c36ce0eb2 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:20.196894 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ef8695-e39e-4e62-8705-987544f935d6 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:20.226200 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:20.231443 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d092cc-ee0e-463a-8b5e-5b61bf1b39e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:20.232249 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d092cc-ee0e-463a-8b5e-5b61bf1b39e6 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:20.232537 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d092cc-ee0e-463a-8b5e-5b61bf1b39e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:20.232914 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d092cc-ee0e-463a-8b5e-5b61bf1b39e6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:20.243868 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d092cc-ee0e-463a-8b5e-5b61bf1b39e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:35:20.251555 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-48d092cc-ee0e-463a-8b5e-5b61bf1b39e6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:35:20.251555 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:20.251555 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:20.251555 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:20.251555 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:20.251555 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:20.251555 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:20.251555 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:20.251555 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:20.251555 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:20.251555 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:20.251555 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:20.251555 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:35:20.251555 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:35:20.251555 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:35:20.251555 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:35:20.252477 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:35:20.252477 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:35:20.252477 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:35:20.252477 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:35:20.252477 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:20.252477 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:20.252477 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:20.252477 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:20.252477 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:20.252477 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:20.252477 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:20.252477 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:20.252477 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:20.252477 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:20.252477 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:20.252477 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:20.253266 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:20.253266 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:20.253266 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:20.253266 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:20.253266 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:35:20.253266 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:35:20.253266 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:35:20.253266 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:35:20.253266 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:35:20.253266 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:35:20.253266 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:35:20.253266 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:20.253266 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 254/501] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 21 22:35:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:20.253917 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d092cc-ee0e-463a-8b5e-5b61bf1b39e6 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:20.274185 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:20.309804 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ef8695-e39e-4e62-8705-987544f935d6 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:20.345055 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40ae6db-7b3a-4bca-9010-1e50c6d91d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:20.345848 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ae6db-7b3a-4bca-9010-1e50c6d91d60 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:20.346134 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ae6db-7b3a-4bca-9010-1e50c6d91d60 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:20.346408 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ae6db-7b3a-4bca-9010-1e50c6d91d60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:20.356872 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d092cc-ee0e-463a-8b5e-5b61bf1b39e6 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:20.357657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40ae6db-7b3a-4bca-9010-1e50c6d91d60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:20.359124 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40ae6db-7b3a-4bca-9010-1e50c6d91d60 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:20.377798 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-d40ae6db-7b3a-4bca-9010-1e50c6d91d60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:20.377798 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:20.377798 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:20.377798 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:20.377798 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:20.377798 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:20.377798 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:20.377798 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:20.377798 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:20.377798 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:20.377798 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:20.377798 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:20.377798 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:20.377798 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:20.377798 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:20.378660 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:20.378660 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:20.378660 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:20.378660 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:20.378660 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:20.378660 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:20.378660 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:20.378660 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:20.378660 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:20.378660 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:20.378660 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:20.378660 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:20.378660 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:20.378660 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:20.378660 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:20.378660 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:20.378660 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:20.379547 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:20.379547 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:20.379547 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:20.379547 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:20.379547 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 248/502] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:20.379805 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40ae6db-7b3a-4bca-9010-1e50c6d91d60 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:20.390830 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f496f98-07c9-4f76-a9a6-304bc2cd9161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:20.391614 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f496f98-07c9-4f76-a9a6-304bc2cd9161 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:20.391895 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f496f98-07c9-4f76-a9a6-304bc2cd9161 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:20.392167 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f496f98-07c9-4f76-a9a6-304bc2cd9161 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:20.402499 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f496f98-07c9-4f76-a9a6-304bc2cd9161 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:20.404264 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f496f98-07c9-4f76-a9a6-304bc2cd9161 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:20.424973 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-4f496f98-07c9-4f76-a9a6-304bc2cd9161 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:20.424973 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:20.424973 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:20.424973 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:20.424973 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:20.424973 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:20.424973 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:20.424973 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:20.424973 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:20.424973 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:20.424973 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:20.424973 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:20.424973 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:20.424973 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:20.424973 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:20.425929 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:20.425929 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:20.425929 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:20.425929 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:20.425929 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:20.425929 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:20.425929 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:20.425929 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:20.425929 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:20.425929 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:20.425929 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:20.425929 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:20.425929 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:20.425929 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:20.425929 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:20.425929 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:20.425929 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:20.426822 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:20.426822 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:20.426822 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:20.426822 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:20.426822 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 255/503] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:20.427108 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f496f98-07c9-4f76-a9a6-304bc2cd9161 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:20.465640 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:20.487490 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40ae6db-7b3a-4bca-9010-1e50c6d91d60 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:20.498861 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:20.521260 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a8b4f9-da46-4ef8-b020-e8e3d1ef3b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:20.521918 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a8b4f9-da46-4ef8-b020-e8e3d1ef3b74 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:20.522202 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a8b4f9-da46-4ef8-b020-e8e3d1ef3b74 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:20.522477 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a8b4f9-da46-4ef8-b020-e8e3d1ef3b74 None admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:20.523786 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a8b4f9-da46-4ef8-b020-e8e3d1ef3b74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:20.524090 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a8b4f9-da46-4ef8-b020-e8e3d1ef3b74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:20.525471 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a8b4f9-da46-4ef8-b020-e8e3d1ef3b74 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:20.528911 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f496f98-07c9-4f76-a9a6-304bc2cd9161 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:20.536956 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 249/504] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:20.538271 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd69f7c1-1b5b-4ed3-8a82-cba5a5f51d8b None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:20.538560 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd69f7c1-1b5b-4ed3-8a82-cba5a5f51d8b None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:20.539520 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd69f7c1-1b5b-4ed3-8a82-cba5a5f51d8b None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:20.540750 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 250/505] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:20.541422 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd69f7c1-1b5b-4ed3-8a82-cba5a5f51d8b None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:20.568711 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f05733-5354-41e9-81f9-1cb6a721d4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:20.569419 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f05733-5354-41e9-81f9-1cb6a721d4a3 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:20.569696 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f05733-5354-41e9-81f9-1cb6a721d4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:20.570485 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f05733-5354-41e9-81f9-1cb6a721d4a3 None admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:20.571255 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f05733-5354-41e9-81f9-1cb6a721d4a3 None admin] RBAC: Adding query filter params (name=service) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:20.571557 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f05733-5354-41e9-81f9-1cb6a721d4a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:20.572924 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f05733-5354-41e9-81f9-1cb6a721d4a3 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:20.580689 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 256/506] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 21 22:35:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:20.581465 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f05733-5354-41e9-81f9-1cb6a721d4a3 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:20.645094 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd69f7c1-1b5b-4ed3-8a82-cba5a5f51d8b None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:20.679255 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69bb49b-e7b6-4df8-8fb4-3e7eda37c632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:20.679913 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69bb49b-e7b6-4df8-8fb4-3e7eda37c632 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:20.680196 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69bb49b-e7b6-4df8-8fb4-3e7eda37c632 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:20.680469 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69bb49b-e7b6-4df8-8fb4-3e7eda37c632 None admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:20.680678 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f05733-5354-41e9-81f9-1cb6a721d4a3 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:20.681184 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69bb49b-e7b6-4df8-8fb4-3e7eda37c632 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:20.681492 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69bb49b-e7b6-4df8-8fb4-3e7eda37c632 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:20.682888 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69bb49b-e7b6-4df8-8fb4-3e7eda37c632 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:20.690935 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:20.694422 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 251/507] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:20.695866 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69bb49b-e7b6-4df8-8fb4-3e7eda37c632 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:20.714035 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:20.718865 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbfcb54-4a84-4015-8780-7fd7e91c56cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:20.719518 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbfcb54-4a84-4015-8780-7fd7e91c56cf None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:20.719798 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbfcb54-4a84-4015-8780-7fd7e91c56cf None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:20.720112 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbfcb54-4a84-4015-8780-7fd7e91c56cf None admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:20.720832 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbfcb54-4a84-4015-8780-7fd7e91c56cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:20.721136 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbfcb54-4a84-4015-8780-7fd7e91c56cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:20.722506 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbfcb54-4a84-4015-8780-7fd7e91c56cf None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:20.734364 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 257/508] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:20.735603 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3e532-1a6f-4d33-9b66-587f6822f747 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:20.735888 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3e532-1a6f-4d33-9b66-587f6822f747 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:20.736162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3e532-1a6f-4d33-9b66-587f6822f747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:20.751815 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-0da3e532-1a6f-4d33-9b66-587f6822f747 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:20.753526 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-0da3e532-1a6f-4d33-9b66-587f6822f747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:20.798256 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69bb49b-e7b6-4df8-8fb4-3e7eda37c632 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:20.819362 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 258/509] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:35:20.820742 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da3e532-1a6f-4d33-9b66-587f6822f747 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:20.843568 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc72079-1b35-412c-8b6d-e52be3dd42ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:20.844516 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc72079-1b35-412c-8b6d-e52be3dd42ed None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:20.844794 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc72079-1b35-412c-8b6d-e52be3dd42ed None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:20.845375 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc72079-1b35-412c-8b6d-e52be3dd42ed None admin] PATH_INFO: `/v3/users/nova` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:20.853770 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc72079-1b35-412c-8b6d-e52be3dd42ed None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:20.855224 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc72079-1b35-412c-8b6d-e52be3dd42ed None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:20.869363 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-dfc72079-1b35-412c-8b6d-e52be3dd42ed None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:35:20.869363 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:20.869363 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:20.869363 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:20.869363 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:20.869363 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:20.869363 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:20.869363 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:20.869363 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:20.869363 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:20.869363 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:20.869363 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:20.869363 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:35:20.869363 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:35:20.869363 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:35:20.869363 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:35:20.870239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:20.870239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:20.870239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:35:20.870239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:35:20.870239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:35:20.870239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:35:20.870239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:20.870239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:20.870239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:20.870239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:20.870239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:20.870239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:20.870239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:20.870239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:20.870239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:20.870239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:20.870239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:20.870239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:20.871170 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:20.871170 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:20.871170 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:35:20.871170 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:35:20.871170 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:35:20.871170 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:35:20.871170 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:35:20.871170 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:35:20.871170 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:35:20.871170 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:20.871170 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 252/510] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Jul 21 22:35:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:20.871729 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc72079-1b35-412c-8b6d-e52be3dd42ed None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:20.921514 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da3e532-1a6f-4d33-9b66-587f6822f747 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:20.928381 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:20.943559 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:20.954467 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d64f9-8530-4c58-87a1-f031a8391bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:20.955299 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d64f9-8530-4c58-87a1-f031a8391bf9 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:20.955580 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d64f9-8530-4c58-87a1-f031a8391bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:20.955851 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d64f9-8530-4c58-87a1-f031a8391bf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:20.966339 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743d64f9-8530-4c58-87a1-f031a8391bf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:20.967786 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743d64f9-8530-4c58-87a1-f031a8391bf9 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:20.969870 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc72079-1b35-412c-8b6d-e52be3dd42ed None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:20.985516 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-743d64f9-8530-4c58-87a1-f031a8391bf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:20.985516 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:20.985516 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:20.985516 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:20.985516 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:20.985516 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:20.985516 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:20.985516 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:20.985516 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:20.985516 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:20.985516 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:20.985516 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:20.985516 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:20.985516 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:20.985516 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:20.986517 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:20.986517 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:20.986517 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:20.986517 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:20.986517 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:20.986517 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:20.986517 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:20.986517 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:20.986517 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:20.986517 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:20.986517 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:20.986517 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:20.986517 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:20.986517 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:20.986517 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:20.986517 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:20.986517 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:20.987428 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:20.987428 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:20.987428 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:20.987428 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:20.987428 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 259/511] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:20.988696 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d64f9-8530-4c58-87a1-f031a8391bf9 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:21.004752 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967710b8-218a-4186-b8a6-b960e2b49878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:21.005484 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967710b8-218a-4186-b8a6-b960e2b49878 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:21.005757 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967710b8-218a-4186-b8a6-b960e2b49878 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:21.006021 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967710b8-218a-4186-b8a6-b960e2b49878 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:21.013385 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967710b8-218a-4186-b8a6-b960e2b49878 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:21.014834 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967710b8-218a-4186-b8a6-b960e2b49878 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:21.028984 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-967710b8-218a-4186-b8a6-b960e2b49878 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 22:35:21.028984 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:21.028984 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:21.028984 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:21.028984 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:21.028984 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:21.028984 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:21.028984 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:21.028984 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:21.028984 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:21.028984 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:21.028984 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:21.028984 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:35:21.028984 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:35:21.028984 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:35:21.028984 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:35:21.030079 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:21.030079 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:21.030079 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:35:21.030079 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:35:21.030079 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:35:21.030079 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:35:21.030079 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:21.030079 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:21.030079 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:21.030079 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:21.030079 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:21.030079 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:21.030079 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:21.030079 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:21.030079 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:21.030079 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:21.030079 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:21.030079 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:21.031224 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:21.031224 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:21.031224 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:35:21.031224 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:35:21.031224 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:35:21.031224 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:35:21.031224 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:35:21.031224 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:35:21.031224 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 22:35:21.031224 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:21.031224 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 253/512] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 21 22:35:20 2022] GET /identity/v3/users/neutron => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:21.031895 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967710b8-218a-4186-b8a6-b960e2b49878 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:21.089027 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d64f9-8530-4c58-87a1-f031a8391bf9 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:21.122410 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32989688-88d4-43e3-a7bc-42e3d5756810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:21.123495 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32989688-88d4-43e3-a7bc-42e3d5756810 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:21.123495 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32989688-88d4-43e3-a7bc-42e3d5756810 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:21.123784 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32989688-88d4-43e3-a7bc-42e3d5756810 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:21.128783 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967710b8-218a-4186-b8a6-b960e2b49878 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:21.130395 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32989688-88d4-43e3-a7bc-42e3d5756810 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:21.131874 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32989688-88d4-43e3-a7bc-42e3d5756810 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:21.147761 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:21.153057 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-32989688-88d4-43e3-a7bc-42e3d5756810 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:35:21.153057 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:21.153057 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:21.153057 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:21.153057 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:21.153057 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:21.153057 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:21.153057 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:21.153057 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:21.153057 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:21.153057 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:21.153057 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:21.153057 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:35:21.153057 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:35:21.153057 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:35:21.153916 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:35:21.153916 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:21.153916 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:21.153916 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:21.153916 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:21.153916 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:21.153916 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:21.153916 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:21.153916 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:21.153916 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:21.153916 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:21.153916 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:21.153916 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:21.153916 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:21.153916 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:21.153916 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:21.153916 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:21.154982 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:35:21.154982 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:35:21.154982 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:35:21.154982 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:35:21.154982 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:35:21.154982 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:35:21.154982 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:35:21.154982 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:21.154982 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 260/513] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:35:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:21.155881 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32989688-88d4-43e3-a7bc-42e3d5756810 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:21.167451 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6419a-ed7c-4c5c-a92e-9b273942a4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:21.168246 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6419a-ed7c-4c5c-a92e-9b273942a4be None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:21.168536 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6419a-ed7c-4c5c-a92e-9b273942a4be None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:21.168814 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6419a-ed7c-4c5c-a92e-9b273942a4be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:21.179354 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c6419a-ed7c-4c5c-a92e-9b273942a4be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:21.180791 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c6419a-ed7c-4c5c-a92e-9b273942a4be None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:21.188251 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:21.197900 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-a5c6419a-ed7c-4c5c-a92e-9b273942a4be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:21.197900 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:21.197900 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:21.197900 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:21.197900 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:21.197900 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:21.197900 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:21.197900 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:21.197900 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:21.197900 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:21.197900 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:21.197900 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:21.197900 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:21.197900 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:21.197900 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:21.203592 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:21.203592 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:21.203592 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:21.203592 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:21.203592 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:21.203592 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:21.203592 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:21.203592 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:21.203592 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:21.203592 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:21.203592 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:21.203592 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:21.203592 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:21.203592 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:21.203592 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:21.203592 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:21.203592 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:21.204442 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:21.204442 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:21.204442 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:21.204442 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:21.204442 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 254/514] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:21.204442 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6419a-ed7c-4c5c-a92e-9b273942a4be None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:21.256971 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32989688-88d4-43e3-a7bc-42e3d5756810 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:21.292671 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c56c0-fe88-4349-8eb1-d01da72091bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:21.293337 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c56c0-fe88-4349-8eb1-d01da72091bf None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:21.293616 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c56c0-fe88-4349-8eb1-d01da72091bf None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:21.293894 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c56c0-fe88-4349-8eb1-d01da72091bf None admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:21.294721 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9c56c0-fe88-4349-8eb1-d01da72091bf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:21.295056 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9c56c0-fe88-4349-8eb1-d01da72091bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:21.296506 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9c56c0-fe88-4349-8eb1-d01da72091bf None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:21.298998 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6419a-ed7c-4c5c-a92e-9b273942a4be None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:21.331656 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 261/515] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Jul 21 22:35:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:21.332422 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c56c0-fe88-4349-8eb1-d01da72091bf None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:21.336222 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2083f08e-5f58-4708-8cbd-e0f1e72fc8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:21.337758 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2083f08e-5f58-4708-8cbd-e0f1e72fc8fc None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:21.338048 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2083f08e-5f58-4708-8cbd-e0f1e72fc8fc None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:21.338329 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2083f08e-5f58-4708-8cbd-e0f1e72fc8fc None admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:21.339086 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2083f08e-5f58-4708-8cbd-e0f1e72fc8fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:21.339397 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2083f08e-5f58-4708-8cbd-e0f1e72fc8fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:21.340756 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2083f08e-5f58-4708-8cbd-e0f1e72fc8fc None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:21.353900 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 255/516] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:21.354688 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2083f08e-5f58-4708-8cbd-e0f1e72fc8fc None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:21.385327 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:21.426493 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:21.435041 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c56c0-fe88-4349-8eb1-d01da72091bf None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:21.453907 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2083f08e-5f58-4708-8cbd-e0f1e72fc8fc None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:21.469789 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc440bb-b02c-4af6-947b-5a8f8ac12b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:21.470458 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc440bb-b02c-4af6-947b-5a8f8ac12b45 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:21.470745 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc440bb-b02c-4af6-947b-5a8f8ac12b45 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:21.471057 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc440bb-b02c-4af6-947b-5a8f8ac12b45 None admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:21.471899 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc440bb-b02c-4af6-947b-5a8f8ac12b45 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:21.472215 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc440bb-b02c-4af6-947b-5a8f8ac12b45 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:21.473820 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc440bb-b02c-4af6-947b-5a8f8ac12b45 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:21.488308 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2438d160-5168-4513-b680-4360eb315bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:21.488963 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438d160-5168-4513-b680-4360eb315bb4 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:21.489242 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438d160-5168-4513-b680-4360eb315bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:21.489513 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438d160-5168-4513-b680-4360eb315bb4 None admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:21.490294 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2438d160-5168-4513-b680-4360eb315bb4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:21.490599 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2438d160-5168-4513-b680-4360eb315bb4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:21.492080 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2438d160-5168-4513-b680-4360eb315bb4 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:21.493666 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 262/517] 10.222.0.52 () {56 vars in 1139 bytes} [Thu Jul 21 22:35:21 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:21.494516 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc440bb-b02c-4af6-947b-5a8f8ac12b45 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:21.514715 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 256/518] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 22:35:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:21.515566 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2438d160-5168-4513-b680-4360eb315bb4 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:21.631243 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc440bb-b02c-4af6-947b-5a8f8ac12b45 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:21.658798 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:21.665036 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc66964-1109-4dfd-bcbe-4ac8abed5543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:21.665695 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc66964-1109-4dfd-bcbe-4ac8abed5543 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:21.665977 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc66964-1109-4dfd-bcbe-4ac8abed5543 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:21.666278 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc66964-1109-4dfd-bcbe-4ac8abed5543 None admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:21.666983 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2438d160-5168-4513-b680-4360eb315bb4 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:21.667082 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc66964-1109-4dfd-bcbe-4ac8abed5543 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:21.667389 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc66964-1109-4dfd-bcbe-4ac8abed5543 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:21.673624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc66964-1109-4dfd-bcbe-4ac8abed5543 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:21.680605 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 263/519] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:21.681396 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc66964-1109-4dfd-bcbe-4ac8abed5543 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:21.693732 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:21.703023 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a102ea86-3ac3-4c22-8511-3bc7e8828021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:21.703650 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102ea86-3ac3-4c22-8511-3bc7e8828021 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:21.703930 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102ea86-3ac3-4c22-8511-3bc7e8828021 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:21.704201 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102ea86-3ac3-4c22-8511-3bc7e8828021 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:21.772411 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a102ea86-3ac3-4c22-8511-3bc7e8828021 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:21.773789 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a102ea86-3ac3-4c22-8511-3bc7e8828021 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:21.783672 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc66964-1109-4dfd-bcbe-4ac8abed5543 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:21.819417 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d068081-e790-404d-a05c-878c54ff88d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:21.820926 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d068081-e790-404d-a05c-878c54ff88d8 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:21.821212 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d068081-e790-404d-a05c-878c54ff88d8 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:21.821497 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d068081-e790-404d-a05c-878c54ff88d8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:21.830682 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d068081-e790-404d-a05c-878c54ff88d8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:21.832373 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d068081-e790-404d-a05c-878c54ff88d8 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:21.848709 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-1d068081-e790-404d-a05c-878c54ff88d8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 22:35:21.848709 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:21.848709 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:21.848709 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:21.848709 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:21.848709 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:21.848709 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:21.848709 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:21.848709 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:21.848709 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:21.848709 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:21.848709 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:21.848709 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:35:21.848709 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:35:21.848709 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:35:21.848709 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:35:21.849606 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:21.849606 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:21.849606 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:35:21.849606 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:35:21.849606 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:35:21.849606 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:35:21.849606 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:21.849606 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:21.849606 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:21.849606 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:21.849606 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:21.849606 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:21.849606 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:21.849606 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:21.849606 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:21.849606 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:21.849606 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:21.849606 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:21.850484 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:21.850484 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:21.850484 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:35:21.850484 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:35:21.850484 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:35:21.850484 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:35:21.850484 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:35:21.850484 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:35:21.850484 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 22:35:21.850484 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:21.850484 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 264/520] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:35:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:21.851225 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d068081-e790-404d-a05c-878c54ff88d8 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:21.865232 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:21.875190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 257/521] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:35:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:35:21.875922 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a102ea86-3ac3-4c22-8511-3bc7e8828021 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:21.923381 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:21.953885 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d068081-e790-404d-a05c-878c54ff88d8 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:21.972429 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a102ea86-3ac3-4c22-8511-3bc7e8828021 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:21.987295 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccccee7-603f-4063-9fb1-6be44566589d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:21.987967 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccccee7-603f-4063-9fb1-6be44566589d None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:21.988272 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccccee7-603f-4063-9fb1-6be44566589d None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:21.988542 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccccee7-603f-4063-9fb1-6be44566589d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:22.005858 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cf2331-6886-4909-b735-9a61a837da27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:22.006496 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf2331-6886-4909-b735-9a61a837da27 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:22.006801 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf2331-6886-4909-b735-9a61a837da27 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:22.007123 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf2331-6886-4909-b735-9a61a837da27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:22.008495 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cf2331-6886-4909-b735-9a61a837da27 None admin] RBAC: Adding query filter params (role.id=715d866890d14923b11470d544b4f960, scope.project.id=aa11ab8229b642ce962193b74001d3bc, user.id=927778d005fd48a4b4b8fd3c4419bb36) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:22.008803 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cf2331-6886-4909-b735-9a61a837da27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:22.010201 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cf2331-6886-4909-b735-9a61a837da27 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:22.015501 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 258/522] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 22:35:21 2022] GET /identity/v3/role_assignments?user.id=927778d005fd48a4b4b8fd3c4419bb36&scope.project.id=aa11ab8229b642ce962193b74001d3bc&role.id=715d866890d14923b11470d544b4f960 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:22.016236 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cf2331-6886-4909-b735-9a61a837da27 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:22.064953 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccccee7-603f-4063-9fb1-6be44566589d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:22.066617 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccccee7-603f-4063-9fb1-6be44566589d None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:22.079083 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:22.130084 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cf2331-6886-4909-b735-9a61a837da27 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:22.147878 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:22.173890 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcfc86c-8410-4727-a0dd-bfed9f20539d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:22.174568 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcfc86c-8410-4727-a0dd-bfed9f20539d None admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:22.174893 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcfc86c-8410-4727-a0dd-bfed9f20539d None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:22.175171 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcfc86c-8410-4727-a0dd-bfed9f20539d None admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:22.176744 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcfc86c-8410-4727-a0dd-bfed9f20539d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:22.178368 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcfc86c-8410-4727-a0dd-bfed9f20539d None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:22.184768 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 265/523] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:35:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:35:22.185917 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccccee7-603f-4063-9fb1-6be44566589d None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:22.265148 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 259/524] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 21 22:35:22 2022] POST /identity/v3/users => generated 247 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:22.266011 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcfc86c-8410-4727-a0dd-bfed9f20539d None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:22.311472 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccccee7-603f-4063-9fb1-6be44566589d None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:22.357419 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075cefb3-1ef4-4dd4-a68f-30672dc09146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:22.358142 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075cefb3-1ef4-4dd4-a68f-30672dc09146 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:22.358482 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075cefb3-1ef4-4dd4-a68f-30672dc09146 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:22.358787 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075cefb3-1ef4-4dd4-a68f-30672dc09146 None admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:22.359689 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075cefb3-1ef4-4dd4-a68f-30672dc09146 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:22.359999 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075cefb3-1ef4-4dd4-a68f-30672dc09146 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:22.361496 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075cefb3-1ef4-4dd4-a68f-30672dc09146 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:22.386275 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 266/525] 10.222.0.52 () {56 vars in 1137 bytes} [Thu Jul 21 22:35:22 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:22.387517 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075cefb3-1ef4-4dd4-a68f-30672dc09146 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:22.417265 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcfc86c-8410-4727-a0dd-bfed9f20539d None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:22.419563 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:22.433762 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:22.460521 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d5cc24-a0a4-4529-8a18-20fd87c7fc30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:22.460521 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5cc24-a0a4-4529-8a18-20fd87c7fc30 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:22.460825 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5cc24-a0a4-4529-8a18-20fd87c7fc30 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:22.461087 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5cc24-a0a4-4529-8a18-20fd87c7fc30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:22.480424 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d5cc24-a0a4-4529-8a18-20fd87c7fc30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:22.482210 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d5cc24-a0a4-4529-8a18-20fd87c7fc30 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:22.510081 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-19d5cc24-a0a4-4529-8a18-20fd87c7fc30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:22.510081 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:22.510081 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:22.510081 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:22.510081 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:22.510081 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:22.510081 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:22.510081 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:22.510081 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:22.510081 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:22.510081 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:22.510081 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:22.510081 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:22.510081 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:22.510081 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:22.511216 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:22.511216 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:22.511216 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:22.511216 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:22.511216 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:22.511216 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:22.511216 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:22.511216 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:22.511216 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:22.511216 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:22.511216 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:22.511216 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:22.511216 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:22.511216 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:22.511216 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:22.511216 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:22.511216 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:22.512171 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:22.512171 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:22.512171 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:22.512171 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:22.512171 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 260/526] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:22.513232 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d5cc24-a0a4-4529-8a18-20fd87c7fc30 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:22.530008 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075cefb3-1ef4-4dd4-a68f-30672dc09146 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:22.571310 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d0ed00-f4e3-4a70-b9aa-823e07b02e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:22.572283 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d0ed00-f4e3-4a70-b9aa-823e07b02e35 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:22.572642 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d0ed00-f4e3-4a70-b9aa-823e07b02e35 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:22.572993 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d0ed00-f4e3-4a70-b9aa-823e07b02e35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:22.584301 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d0ed00-f4e3-4a70-b9aa-823e07b02e35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:22.586386 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d0ed00-f4e3-4a70-b9aa-823e07b02e35 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:22.607129 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-26d0ed00-f4e3-4a70-b9aa-823e07b02e35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:22.607129 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:22.607129 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:22.607129 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:22.607129 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:22.607129 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:22.607129 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:22.607129 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:22.607129 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:22.607129 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:22.607129 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:22.607129 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:22.607129 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:22.607129 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:22.607129 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:22.608092 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:22.608092 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:22.608092 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:22.608092 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:22.608092 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:22.608092 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:22.608092 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:22.608092 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:22.608092 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:22.608092 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:22.608092 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:22.608092 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:22.608092 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:22.608092 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:22.608092 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:22.608092 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:22.608092 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:22.608949 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:22.608949 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:22.608949 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:22.608949 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:22.608949 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 267/527] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:22.609955 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d0ed00-f4e3-4a70-b9aa-823e07b02e35 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:22.649121 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d5cc24-a0a4-4529-8a18-20fd87c7fc30 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:22.700888 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d00699-817e-4f50-b621-5a32d9faa4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:22.702273 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d00699-817e-4f50-b621-5a32d9faa4c8 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:22.702926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d00699-817e-4f50-b621-5a32d9faa4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:22.703480 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d00699-817e-4f50-b621-5a32d9faa4c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:22.705341 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:22.728026 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d0ed00-f4e3-4a70-b9aa-823e07b02e35 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:22.753410 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:22.767213 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602983d5-ac5e-4dd6-a9b4-b76508b5a7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:22.768039 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602983d5-ac5e-4dd6-a9b4-b76508b5a7f7 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:22.768399 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602983d5-ac5e-4dd6-a9b4-b76508b5a7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:22.768751 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602983d5-ac5e-4dd6-a9b4-b76508b5a7f7 None admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:22.769566 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602983d5-ac5e-4dd6-a9b4-b76508b5a7f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:22.769950 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602983d5-ac5e-4dd6-a9b4-b76508b5a7f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:22.771843 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602983d5-ac5e-4dd6-a9b4-b76508b5a7f7 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:22.792642 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 268/528] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:22.793639 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602983d5-ac5e-4dd6-a9b4-b76508b5a7f7 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:22.804489 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d00699-817e-4f50-b621-5a32d9faa4c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:22.806789 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d00699-817e-4f50-b621-5a32d9faa4c8 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:22.921854 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602983d5-ac5e-4dd6-a9b4-b76508b5a7f7 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:22.936724 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 261/529] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:35:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:35:22.942070 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d00699-817e-4f50-b621-5a32d9faa4c8 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:22.971737 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c009a74b-fc1c-4e76-8f00-d00cd82bce19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:22.973314 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c009a74b-fc1c-4e76-8f00-d00cd82bce19 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:22.973806 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c009a74b-fc1c-4e76-8f00-d00cd82bce19 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:22.974271 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c009a74b-fc1c-4e76-8f00-d00cd82bce19 None admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:22.975800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c009a74b-fc1c-4e76-8f00-d00cd82bce19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:22.976493 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c009a74b-fc1c-4e76-8f00-d00cd82bce19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:22.978720 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c009a74b-fc1c-4e76-8f00-d00cd82bce19 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:22.990410 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:22.995617 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 269/530] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:22.996575 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c009a74b-fc1c-4e76-8f00-d00cd82bce19 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:23.020391 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:23.060813 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d00699-817e-4f50-b621-5a32d9faa4c8 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:23.099624 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23daa392-2a39-4960-846b-52254760196a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:23.100791 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23daa392-2a39-4960-846b-52254760196a None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:23.101033 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23daa392-2a39-4960-846b-52254760196a None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:23.101404 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23daa392-2a39-4960-846b-52254760196a None admin] PATH_INFO: `/v3/projects/service` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:23.109203 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23daa392-2a39-4960-846b-52254760196a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:23.110749 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23daa392-2a39-4960-846b-52254760196a None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:23.125980 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-23daa392-2a39-4960-846b-52254760196a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:35:23.125980 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:23.125980 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:23.125980 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:23.125980 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:23.125980 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:23.125980 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:23.125980 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:23.125980 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:23.125980 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:23.125980 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:23.125980 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:23.125980 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:35:23.125980 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:35:23.125980 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:35:23.127094 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:35:23.127094 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:23.127094 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:23.127094 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:23.127094 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:23.127094 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:23.127094 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:23.127094 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:23.127094 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:23.127094 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:23.127094 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:23.127094 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:23.127094 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:23.127094 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:23.127094 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:23.127094 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:23.127094 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:23.128139 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:35:23.128139 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:35:23.128139 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:35:23.128139 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:35:23.128139 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:35:23.128139 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:35:23.128139 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:35:23.128139 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:23.129937 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 262/531] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:35:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:23.130833 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23daa392-2a39-4960-846b-52254760196a None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:23.131083 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c009a74b-fc1c-4e76-8f00-d00cd82bce19 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:23.180914 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0f4fe7-e0b8-4b7c-b3c0-86a91c89856d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:23.182285 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f4fe7-e0b8-4b7c-b3c0-86a91c89856d None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:23.182605 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f4fe7-e0b8-4b7c-b3c0-86a91c89856d None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:23.183291 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f4fe7-e0b8-4b7c-b3c0-86a91c89856d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:23.193975 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0f4fe7-e0b8-4b7c-b3c0-86a91c89856d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:23.196089 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0f4fe7-e0b8-4b7c-b3c0-86a91c89856d None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:23.217237 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-db0f4fe7-e0b8-4b7c-b3c0-86a91c89856d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:23.217237 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:23.217237 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:23.217237 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:23.217237 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:23.217237 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:23.217237 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:23.217237 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:23.217237 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:23.217237 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:23.217237 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:23.217237 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:23.217237 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:23.217237 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:23.217237 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:23.218134 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:23.218134 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:23.218134 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:23.218134 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:23.218134 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:23.218134 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:23.218134 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:23.218134 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:23.218134 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:23.218134 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:23.218134 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:23.218134 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:23.218134 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:23.218134 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:23.218134 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:23.218134 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:23.218134 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:23.219334 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:23.219334 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:23.219334 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:23.219334 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:23.219334 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 270/532] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:23.219590 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0f4fe7-e0b8-4b7c-b3c0-86a91c89856d None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:23.251308 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23daa392-2a39-4960-846b-52254760196a None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:23.279291 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:23.290075 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:23.291469 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae8f43-a5f1-4631-96b9-791fe1a766b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:23.292333 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae8f43-a5f1-4631-96b9-791fe1a766b3 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:23.292893 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae8f43-a5f1-4631-96b9-791fe1a766b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:23.293174 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae8f43-a5f1-4631-96b9-791fe1a766b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:23.302572 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ae8f43-a5f1-4631-96b9-791fe1a766b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:23.304104 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ae8f43-a5f1-4631-96b9-791fe1a766b3 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:23.325408 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-93ae8f43-a5f1-4631-96b9-791fe1a766b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:35:23.325408 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:23.325408 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:23.325408 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:23.325408 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:23.325408 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:23.325408 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:23.325408 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:23.325408 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:23.325408 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:23.325408 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:23.325408 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:23.325408 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:35:23.325408 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:35:23.325408 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:35:23.327010 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:35:23.327010 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:23.327010 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:23.327010 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:23.327010 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:23.327010 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:23.327010 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:23.327010 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:23.327010 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:23.327010 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:23.327010 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:23.327010 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:23.327010 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:23.327010 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:23.327010 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:23.327010 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:23.327010 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:23.329321 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:35:23.329321 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:35:23.329321 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:35:23.329321 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:35:23.329321 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:35:23.329321 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:35:23.329321 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:35:23.329321 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:23.329321 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 263/533] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:35:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:23.329321 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae8f43-a5f1-4631-96b9-791fe1a766b3 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:23.349653 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0f4fe7-e0b8-4b7c-b3c0-86a91c89856d None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:23.388466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa43e282-01d1-42db-a905-843b457c4c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:23.389130 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43e282-01d1-42db-a905-843b457c4c1c None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:23.389412 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43e282-01d1-42db-a905-843b457c4c1c None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:23.389726 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43e282-01d1-42db-a905-843b457c4c1c None admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:23.390630 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa43e282-01d1-42db-a905-843b457c4c1c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:23.391078 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa43e282-01d1-42db-a905-843b457c4c1c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:23.392571 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa43e282-01d1-42db-a905-843b457c4c1c None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:23.413247 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 271/534] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 22:35:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:23.415489 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa43e282-01d1-42db-a905-843b457c4c1c None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:23.457975 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae8f43-a5f1-4631-96b9-791fe1a766b3 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:23.505152 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd3bc6-0305-4c62-a0c6-3597cc4590cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:23.506113 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd3bc6-0305-4c62-a0c6-3597cc4590cf None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:23.506598 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd3bc6-0305-4c62-a0c6-3597cc4590cf None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:23.507306 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd3bc6-0305-4c62-a0c6-3597cc4590cf None admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:23.508239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bd3bc6-0305-4c62-a0c6-3597cc4590cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:23.508755 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bd3bc6-0305-4c62-a0c6-3597cc4590cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:23.510318 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bd3bc6-0305-4c62-a0c6-3597cc4590cf None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:23.524249 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 264/535] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:23.525440 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd3bc6-0305-4c62-a0c6-3597cc4590cf None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:23.556105 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa43e282-01d1-42db-a905-843b457c4c1c None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:23.574279 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:23.579407 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:23.597322 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c569557-d2ee-426e-9830-eaf52533230c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:23.598298 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c569557-d2ee-426e-9830-eaf52533230c None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:23.598846 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c569557-d2ee-426e-9830-eaf52533230c None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:23.599322 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c569557-d2ee-426e-9830-eaf52533230c None admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:23.600303 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c569557-d2ee-426e-9830-eaf52533230c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:23.600859 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c569557-d2ee-426e-9830-eaf52533230c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:23.602742 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c569557-d2ee-426e-9830-eaf52533230c None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:23.616873 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 272/536] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 22:35:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:23.617976 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c569557-d2ee-426e-9830-eaf52533230c None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:23.656297 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd3bc6-0305-4c62-a0c6-3597cc4590cf None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:23.703422 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5284f9c-5adc-403b-b528-9a8ad6c77115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:23.704076 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5284f9c-5adc-403b-b528-9a8ad6c77115 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:23.704359 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5284f9c-5adc-403b-b528-9a8ad6c77115 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:23.704637 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5284f9c-5adc-403b-b528-9a8ad6c77115 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:23.705574 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5284f9c-5adc-403b-b528-9a8ad6c77115 None admin] RBAC: Adding query filter params (role.id=715d866890d14923b11470d544b4f960, scope.project.id=aa11ab8229b642ce962193b74001d3bc, user.id=6bfdb655a2a44b33ab724daaf73a294b) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:23.705889 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5284f9c-5adc-403b-b528-9a8ad6c77115 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:23.707587 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5284f9c-5adc-403b-b528-9a8ad6c77115 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:23.717825 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 265/537] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 22:35:23 2022] GET /identity/v3/role_assignments?user.id=6bfdb655a2a44b33ab724daaf73a294b&scope.project.id=aa11ab8229b642ce962193b74001d3bc&role.id=715d866890d14923b11470d544b4f960 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:23.718628 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5284f9c-5adc-403b-b528-9a8ad6c77115 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:23.738552 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c569557-d2ee-426e-9830-eaf52533230c None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:23.781748 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf8a7f2-6ab7-40e4-8d83-a8d25b2878b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:23.783094 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8a7f2-6ab7-40e4-8d83-a8d25b2878b5 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:23.783683 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8a7f2-6ab7-40e4-8d83-a8d25b2878b5 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:23.784161 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8a7f2-6ab7-40e4-8d83-a8d25b2878b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:23.807689 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf8a7f2-6ab7-40e4-8d83-a8d25b2878b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:23.809549 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf8a7f2-6ab7-40e4-8d83-a8d25b2878b5 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:23.840400 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-dbf8a7f2-6ab7-40e4-8d83-a8d25b2878b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:35:23.840400 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:23.840400 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:23.840400 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:23.840400 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:23.840400 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:23.840400 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:23.840400 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:23.840400 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:23.840400 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:23.840400 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:23.840400 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:23.840400 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:35:23.840400 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:35:23.840400 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:35:23.841314 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:35:23.841314 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:23.841314 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:23.841314 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:23.841314 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:23.841314 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:23.841314 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:23.841314 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:23.841314 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:23.841314 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:23.841314 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:23.841314 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:23.841314 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:23.841314 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:23.841314 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:23.841314 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:23.841314 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:23.842183 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:35:23.842183 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:35:23.842183 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:35:23.842183 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:35:23.842183 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:35:23.842183 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:35:23.842183 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:35:23.842183 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:23.845091 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 273/538] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:35:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:23.853402 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf8a7f2-6ab7-40e4-8d83-a8d25b2878b5 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:23.892779 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:23.964042 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5284f9c-5adc-403b-b528-9a8ad6c77115 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:23.973379 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf8a7f2-6ab7-40e4-8d83-a8d25b2878b5 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:23.975119 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:24.002246 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c6de58-2daf-4bdc-8f0e-256d4296addf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:24.003040 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c6de58-2daf-4bdc-8f0e-256d4296addf None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:24.003396 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c6de58-2daf-4bdc-8f0e-256d4296addf None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:24.003746 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c6de58-2daf-4bdc-8f0e-256d4296addf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:24.004971 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c6de58-2daf-4bdc-8f0e-256d4296addf None admin] RBAC: Adding query filter params (role.id=715d866890d14923b11470d544b4f960, scope.project.id=aa11ab8229b642ce962193b74001d3bc, user.id=48c3e33f24ff4d66a72f06848d769ac7) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:24.005373 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c6de58-2daf-4bdc-8f0e-256d4296addf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:24.007453 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c6de58-2daf-4bdc-8f0e-256d4296addf None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:24.014373 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 266/539] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 22:35:23 2022] GET /identity/v3/role_assignments?user.id=48c3e33f24ff4d66a72f06848d769ac7&scope.project.id=aa11ab8229b642ce962193b74001d3bc&role.id=715d866890d14923b11470d544b4f960 => generated 259 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:24.045027 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d839e1f-b599-4e00-a5f3-5c9b4e4c9cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:24.045957 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d839e1f-b599-4e00-a5f3-5c9b4e4c9cc5 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:24.046419 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d839e1f-b599-4e00-a5f3-5c9b4e4c9cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:24.046879 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d839e1f-b599-4e00-a5f3-5c9b4e4c9cc5 None admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:24.047824 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d839e1f-b599-4e00-a5f3-5c9b4e4c9cc5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:24.048274 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d839e1f-b599-4e00-a5f3-5c9b4e4c9cc5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:24.049865 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d839e1f-b599-4e00-a5f3-5c9b4e4c9cc5 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:24.069481 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 274/540] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 22:35:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:35:24.076025 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c6de58-2daf-4bdc-8f0e-256d4296addf None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:24.261276 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c6de58-2daf-4bdc-8f0e-256d4296addf None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:24.316086 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7201c561-4888-4fc2-8fe3-de5980c7d4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:24.316997 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201c561-4888-4fc2-8fe3-de5980c7d4fc None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:24.317447 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201c561-4888-4fc2-8fe3-de5980c7d4fc None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:24.317864 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201c561-4888-4fc2-8fe3-de5980c7d4fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:24.318888 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7201c561-4888-4fc2-8fe3-de5980c7d4fc None admin] RBAC: Adding query filter params (role.id=715d866890d14923b11470d544b4f960, scope.project.id=aa11ab8229b642ce962193b74001d3bc, user.id=150f42651f754fda83f25fe3b408bce9) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:24.319340 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7201c561-4888-4fc2-8fe3-de5980c7d4fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:24.321544 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7201c561-4888-4fc2-8fe3-de5980c7d4fc None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:24.327791 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 267/541] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 22:35:24 2022] GET /identity/v3/role_assignments?user.id=150f42651f754fda83f25fe3b408bce9&scope.project.id=aa11ab8229b642ce962193b74001d3bc&role.id=715d866890d14923b11470d544b4f960 => generated 259 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:24.419185 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd3ecb-6a31-4943-95a7-6def9bd53871 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:24.420033 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd3ecb-6a31-4943-95a7-6def9bd53871 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:24.420455 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd3ecb-6a31-4943-95a7-6def9bd53871 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:24.422407 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 275/542] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:24.427789 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a2240-ddd4-4750-8998-4c5a873dc533 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:24.428333 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a2240-ddd4-4750-8998-4c5a873dc533 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:24.428850 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a2240-ddd4-4750-8998-4c5a873dc533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:24.455253 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-9e3a2240-ddd4-4750-8998-4c5a873dc533 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:24.456857 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-9e3a2240-ddd4-4750-8998-4c5a873dc533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:24.503849 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:24.560672 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 268/543] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:35:24.583925 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd3ecb-6a31-4943-95a7-6def9bd53871 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:24.622619 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77712c2-ffb1-4e9a-a82c-bb1f0e98017b None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:24.622966 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77712c2-ffb1-4e9a-a82c-bb1f0e98017b None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:24.623217 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77712c2-ffb1-4e9a-a82c-bb1f0e98017b None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:24.624843 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 269/544] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:35:24.630262 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72642852-9166-4171-afe3-f9727afcf65d None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:24.630559 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72642852-9166-4171-afe3-f9727afcf65d None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:24.630884 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72642852-9166-4171-afe3-f9727afcf65d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:24.664425 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-72642852-9166-4171-afe3-f9727afcf65d None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:24.665636 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-72642852-9166-4171-afe3-f9727afcf65d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:24.757127 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 270/545] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:35:24.780157 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:24.784214 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72642852-9166-4171-afe3-f9727afcf65d None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:24.804508 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd3ecb-6a31-4943-95a7-6def9bd53871 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:24.852596 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de3a410-3ebc-42ed-8de6-f9f180adb7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:24.853596 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de3a410-3ebc-42ed-8de6-f9f180adb7f3 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:24.853938 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de3a410-3ebc-42ed-8de6-f9f180adb7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:24.854220 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de3a410-3ebc-42ed-8de6-f9f180adb7f3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:24.866315 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de3a410-3ebc-42ed-8de6-f9f180adb7f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:35:24.882800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-2de3a410-3ebc-42ed-8de6-f9f180adb7f3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:35:24.882800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:24.882800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:24.882800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:24.882800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:24.882800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:24.882800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:24.882800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:24.882800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:24.882800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:24.882800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:24.882800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:24.882800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:35:24.882800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:35:24.882800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:35:24.882800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:35:24.883766 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:35:24.883766 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:35:24.883766 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:35:24.883766 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:35:24.883766 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:24.883766 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:24.883766 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:24.883766 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:24.883766 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:24.883766 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:24.883766 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:24.883766 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:24.883766 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:24.883766 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:24.883766 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:24.883766 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:24.884546 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:24.884546 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:24.884546 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:24.884546 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:24.884546 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:35:24.884546 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:35:24.884546 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:35:24.884546 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:35:24.884546 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:35:24.884546 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:35:24.884546 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:35:24.884546 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:24.884546 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 276/546] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 21 22:35:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:35:24.884546 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de3a410-3ebc-42ed-8de6-f9f180adb7f3 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:25.004546 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de3a410-3ebc-42ed-8de6-f9f180adb7f3 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:25.042599 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684360ab-72f3-4550-9125-decef3aa747c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:25.043483 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684360ab-72f3-4550-9125-decef3aa747c None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:25.044135 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684360ab-72f3-4550-9125-decef3aa747c None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:25.044515 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684360ab-72f3-4550-9125-decef3aa747c None admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:25.048646 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684360ab-72f3-4550-9125-decef3aa747c None admin] RBAC: Adding query filter params (name=service) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:25.048646 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684360ab-72f3-4550-9125-decef3aa747c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:25.048842 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:25.048917 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684360ab-72f3-4550-9125-decef3aa747c None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:25.056678 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72642852-9166-4171-afe3-f9727afcf65d None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:25.074224 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 277/547] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 21 22:35:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:25.075193 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684360ab-72f3-4550-9125-decef3aa747c None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:25.111634 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:25.118019 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f26d32-7fcc-4786-a65f-508b597bba6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:25.119058 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f26d32-7fcc-4786-a65f-508b597bba6e None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:25.119347 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f26d32-7fcc-4786-a65f-508b597bba6e None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:25.119958 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f26d32-7fcc-4786-a65f-508b597bba6e None admin] PATH_INFO: `/v3/roles/service` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:25.131255 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f26d32-7fcc-4786-a65f-508b597bba6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:35:25.140067 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-77f26d32-7fcc-4786-a65f-508b597bba6e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:35:25.140067 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:25.140067 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:25.140067 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:25.140067 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:25.140067 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:25.140067 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:25.140067 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:25.140067 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:25.140067 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:25.140067 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:25.140067 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:25.140067 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:35:25.140067 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:35:25.140067 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:35:25.140067 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:35:25.141011 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:35:25.141011 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:35:25.141011 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:35:25.141011 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:35:25.141011 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:25.141011 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:25.141011 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:25.141011 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:25.141011 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:25.141011 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:25.141011 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:25.141011 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:25.141011 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:25.141011 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:25.141011 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:25.141011 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:25.141790 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:25.141790 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:25.141790 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:25.141790 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:25.141790 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:35:25.141790 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:35:25.141790 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:35:25.141790 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:35:25.141790 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:35:25.141790 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:35:25.141790 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:35:25.141790 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:25.141790 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 271/548] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 21 22:35:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:35:25.147569 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f26d32-7fcc-4786-a65f-508b597bba6e None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:25.259567 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f26d32-7fcc-4786-a65f-508b597bba6e None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:25.296861 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c662784-b4f4-485a-a9a7-3eb8329e6328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:25.297592 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c662784-b4f4-485a-a9a7-3eb8329e6328 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:25.297887 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c662784-b4f4-485a-a9a7-3eb8329e6328 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:25.298173 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c662784-b4f4-485a-a9a7-3eb8329e6328 None admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:25.298978 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c662784-b4f4-485a-a9a7-3eb8329e6328 None admin] RBAC: Adding query filter params (name=service) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:25.299298 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c662784-b4f4-485a-a9a7-3eb8329e6328 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:25.301284 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c662784-b4f4-485a-a9a7-3eb8329e6328 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:25.310297 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 272/549] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 21 22:35:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:35:25.316642 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c662784-b4f4-485a-a9a7-3eb8329e6328 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:25.339144 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:25.370131 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684360ab-72f3-4550-9125-decef3aa747c None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:25.448634 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555330b-5b52-492b-8bcc-4110ee755ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:25.450254 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555330b-5b52-492b-8bcc-4110ee755ddd None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:25.450764 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555330b-5b52-492b-8bcc-4110ee755ddd None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:25.451409 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555330b-5b52-492b-8bcc-4110ee755ddd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:25.475902 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555330b-5b52-492b-8bcc-4110ee755ddd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:25.477633 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555330b-5b52-492b-8bcc-4110ee755ddd None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:25.494323 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c662784-b4f4-485a-a9a7-3eb8329e6328 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:25.516576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-f555330b-5b52-492b-8bcc-4110ee755ddd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:25.516576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:25.516576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:25.516576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:25.516576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:25.516576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:25.516576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:25.516576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:25.516576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:25.516576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:25.516576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:25.516576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:25.516576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:25.516576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:25.516576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:25.517482 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:25.517482 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:25.517482 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:25.517482 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:25.517482 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:25.517482 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:25.517482 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:25.517482 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:25.517482 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:25.517482 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:25.517482 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:25.517482 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:25.517482 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:25.517482 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:25.517482 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:25.517482 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:25.517482 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:25.518365 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:25.518365 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:25.518365 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:25.518365 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:25.522369 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 278/550] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 443 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:25.524214 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555330b-5b52-492b-8bcc-4110ee755ddd None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:25.543140 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90824415-975b-4277-8ef2-902b6eebceb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:25.544660 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90824415-975b-4277-8ef2-902b6eebceb3 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:25.545154 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90824415-975b-4277-8ef2-902b6eebceb3 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:25.545690 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90824415-975b-4277-8ef2-902b6eebceb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:25.570583 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90824415-975b-4277-8ef2-902b6eebceb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:25.572135 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90824415-975b-4277-8ef2-902b6eebceb3 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:25.600357 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-90824415-975b-4277-8ef2-902b6eebceb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:25.600357 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:25.600357 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:25.600357 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:25.600357 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:25.600357 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:25.600357 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:25.600357 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:25.600357 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:25.600357 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:25.600357 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:25.600357 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:25.600357 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:25.600357 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:25.600357 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:25.601374 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:25.601374 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:25.601374 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:25.601374 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:25.601374 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:25.601374 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:25.601374 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:25.601374 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:25.601374 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:25.601374 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:25.601374 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:25.601374 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:25.601374 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:25.601374 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:25.601374 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:25.601374 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:25.601374 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:25.602539 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:25.602539 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:25.602539 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:25.602539 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:25.613519 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 273/551] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:35:25.618908 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90824415-975b-4277-8ef2-902b6eebceb3 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:25.619136 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:25.676324 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555330b-5b52-492b-8bcc-4110ee755ddd None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:25.702155 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:25.712005 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884bb5d8-865e-4251-b90c-00eefc042042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:25.721559 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884bb5d8-865e-4251-b90c-00eefc042042 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:25.721559 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884bb5d8-865e-4251-b90c-00eefc042042 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:25.721559 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884bb5d8-865e-4251-b90c-00eefc042042 None admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:25.721559 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884bb5d8-865e-4251-b90c-00eefc042042 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:25.721559 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884bb5d8-865e-4251-b90c-00eefc042042 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:25.721559 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884bb5d8-865e-4251-b90c-00eefc042042 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:25.728412 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 279/552] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:25.738504 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90824415-975b-4277-8ef2-902b6eebceb3 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:25.748273 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884bb5d8-865e-4251-b90c-00eefc042042 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:25.779753 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f9a16-1d4b-4941-b257-8000154d78d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:25.780909 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f9a16-1d4b-4941-b257-8000154d78d2 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:25.781395 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f9a16-1d4b-4941-b257-8000154d78d2 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:25.781685 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f9a16-1d4b-4941-b257-8000154d78d2 None admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:25.782425 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2f9a16-1d4b-4941-b257-8000154d78d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:25.783084 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2f9a16-1d4b-4941-b257-8000154d78d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:25.784556 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2f9a16-1d4b-4941-b257-8000154d78d2 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:25.799020 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 274/553] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:25.805180 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f9a16-1d4b-4941-b257-8000154d78d2 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:25.946751 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884bb5d8-865e-4251-b90c-00eefc042042 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:25.997421 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:26.001573 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f9a16-1d4b-4941-b257-8000154d78d2 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:26.006603 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4001fd-b366-40ad-b783-3350b4538ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:26.007990 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4001fd-b366-40ad-b783-3350b4538ca4 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:26.008275 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4001fd-b366-40ad-b783-3350b4538ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:26.008550 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4001fd-b366-40ad-b783-3350b4538ca4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:26.016381 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4001fd-b366-40ad-b783-3350b4538ca4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:26.017973 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4001fd-b366-40ad-b783-3350b4538ca4 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:26.032371 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:26.034829 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-4a4001fd-b366-40ad-b783-3350b4538ca4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 21 22:35:26.034829 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:26.034829 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:26.034829 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:26.034829 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:26.034829 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:26.034829 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:26.034829 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:26.034829 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:26.034829 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:26.034829 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:26.034829 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:26.034829 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:35:26.034829 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:35:26.034829 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:35:26.034829 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:35:26.035744 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:26.035744 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:26.035744 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:35:26.035744 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:35:26.035744 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:35:26.035744 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:35:26.035744 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:26.035744 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:26.035744 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:26.035744 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:26.035744 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:26.035744 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:26.035744 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:26.035744 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:26.035744 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:26.035744 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:26.035744 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:26.035744 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:26.037058 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:26.037058 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:26.037058 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:35:26.037058 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:35:26.037058 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:35:26.037058 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:35:26.037058 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:35:26.037058 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:35:26.037058 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 21 22:35:26.037058 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:26.037058 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 280/554] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:35:25 2022] GET /identity/v3/users/glance => generated 84 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:35:26.037628 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3155f5-4182-42f2-9e8c-b7630e2ecbda None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:26.037991 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3155f5-4182-42f2-9e8c-b7630e2ecbda None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:26.038142 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3155f5-4182-42f2-9e8c-b7630e2ecbda None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:26.039711 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 281/555] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:26.044610 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3155f5-4182-42f2-9e8c-b7630e2ecbda None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:26.045555 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecadc53-0efd-4298-abfd-0db2195b347b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:26.046354 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecadc53-0efd-4298-abfd-0db2195b347b None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:26.046637 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecadc53-0efd-4298-abfd-0db2195b347b None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:26.047448 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecadc53-0efd-4298-abfd-0db2195b347b None admin] PATH_INFO: `/v3/users/swift` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:26.067482 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecadc53-0efd-4298-abfd-0db2195b347b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:26.069354 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecadc53-0efd-4298-abfd-0db2195b347b None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:26.091046 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-2ecadc53-0efd-4298-abfd-0db2195b347b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 22:35:26.091046 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:26.091046 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:26.091046 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:26.091046 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:26.091046 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:26.091046 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:26.091046 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:26.091046 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:26.091046 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:26.091046 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:26.091046 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:26.091046 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:35:26.091046 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:35:26.091046 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:35:26.091046 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:35:26.093330 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:26.093330 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:26.093330 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:35:26.093330 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:35:26.093330 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:35:26.093330 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:35:26.093330 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:26.093330 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:26.093330 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:26.093330 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:26.093330 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:26.093330 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:26.093330 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:26.093330 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:26.093330 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:26.093330 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:26.093330 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:26.093330 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:26.095783 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:26.095783 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:26.095783 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:35:26.095783 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:35:26.095783 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:35:26.095783 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:35:26.095783 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:35:26.095783 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:35:26.095783 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 22:35:26.095783 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:26.095783 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 275/556] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 22:35:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:26.095783 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b99f0-af76-4101-a1b1-a2a01e983be7 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:26.095783 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b99f0-af76-4101-a1b1-a2a01e983be7 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:26.096625 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b99f0-af76-4101-a1b1-a2a01e983be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:26.122238 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-9f8b99f0-af76-4101-a1b1-a2a01e983be7 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:26.123810 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-9f8b99f0-af76-4101-a1b1-a2a01e983be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:26.176678 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3155f5-4182-42f2-9e8c-b7630e2ecbda None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:26.214830 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 276/557] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:35:26.215729 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8b99f0-af76-4101-a1b1-a2a01e983be7 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:26.233357 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf365f5-2b83-41c3-a67d-f1adad42588d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:26.234035 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf365f5-2b83-41c3-a67d-f1adad42588d None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:26.234323 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf365f5-2b83-41c3-a67d-f1adad42588d None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:26.234601 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf365f5-2b83-41c3-a67d-f1adad42588d None admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:26.235478 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf365f5-2b83-41c3-a67d-f1adad42588d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:26.236493 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf365f5-2b83-41c3-a67d-f1adad42588d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:26.238039 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf365f5-2b83-41c3-a67d-f1adad42588d None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:26.263814 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 282/558] 10.222.0.52 () {56 vars in 1137 bytes} [Thu Jul 21 22:35:26 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:35:26.264811 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf365f5-2b83-41c3-a67d-f1adad42588d None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:26.313464 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:26.359685 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8b99f0-af76-4101-a1b1-a2a01e983be7 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:26.385432 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:26.391504 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf365f5-2b83-41c3-a67d-f1adad42588d None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:26.401673 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e239f5ca-bec7-4e2d-a8d0-25e627f2b22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:26.402360 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e239f5ca-bec7-4e2d-a8d0-25e627f2b22f None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:26.402647 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e239f5ca-bec7-4e2d-a8d0-25e627f2b22f None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:26.404348 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e239f5ca-bec7-4e2d-a8d0-25e627f2b22f None admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:26.404348 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e239f5ca-bec7-4e2d-a8d0-25e627f2b22f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:26.404348 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e239f5ca-bec7-4e2d-a8d0-25e627f2b22f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:26.405729 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e239f5ca-bec7-4e2d-a8d0-25e627f2b22f None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:26.430742 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 277/559] 10.222.0.52 () {56 vars in 1135 bytes} [Thu Jul 21 22:35:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:26.431807 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e239f5ca-bec7-4e2d-a8d0-25e627f2b22f None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:26.441843 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e823578e-5891-4e93-80fd-383892ef4bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:26.442987 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e823578e-5891-4e93-80fd-383892ef4bdf None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:26.443483 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e823578e-5891-4e93-80fd-383892ef4bdf None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:26.443963 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e823578e-5891-4e93-80fd-383892ef4bdf None admin] PATH_INFO: `/v3/roles/service` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:26.459024 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e823578e-5891-4e93-80fd-383892ef4bdf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:35:26.468624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-e823578e-5891-4e93-80fd-383892ef4bdf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:35:26.468624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:26.468624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:26.468624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:26.468624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:26.468624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:26.468624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:26.468624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:26.468624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:26.468624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:26.468624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:26.468624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:26.468624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:35:26.468624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:35:26.468624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:35:26.468624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:35:26.469790 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:35:26.469790 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:35:26.469790 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:35:26.469790 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:35:26.469790 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:26.469790 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:26.469790 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:26.469790 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:26.469790 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:26.469790 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:26.469790 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:26.469790 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:26.469790 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:26.469790 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:26.469790 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:26.469790 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:26.470643 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:26.470643 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:26.470643 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:26.470643 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:26.470643 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:35:26.470643 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:35:26.470643 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:35:26.470643 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:35:26.470643 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:35:26.470643 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:35:26.470643 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:35:26.470643 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:26.470643 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 283/560] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 21 22:35:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:26.471372 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e823578e-5891-4e93-80fd-383892ef4bdf None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:26.561896 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e239f5ca-bec7-4e2d-a8d0-25e627f2b22f None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:26.577282 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e823578e-5891-4e93-80fd-383892ef4bdf None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:26.580581 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:26.599822 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2e2e4e-bf03-4022-8ebe-afcb7d2561b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:26.600616 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e2e4e-bf03-4022-8ebe-afcb7d2561b0 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:26.601408 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e2e4e-bf03-4022-8ebe-afcb7d2561b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:26.601679 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e2e4e-bf03-4022-8ebe-afcb7d2561b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:26.612904 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe7a39-d004-4946-a41b-262fe7462a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:26.613068 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2e2e4e-bf03-4022-8ebe-afcb7d2561b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:26.613552 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe7a39-d004-4946-a41b-262fe7462a6c None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:26.613838 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe7a39-d004-4946-a41b-262fe7462a6c None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:26.614113 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe7a39-d004-4946-a41b-262fe7462a6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:26.614536 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2e2e4e-bf03-4022-8ebe-afcb7d2561b0 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:26.632805 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-2a2e2e4e-bf03-4022-8ebe-afcb7d2561b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:26.632805 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:26.632805 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:26.632805 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:26.632805 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:26.632805 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:26.632805 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:26.632805 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:26.632805 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:26.632805 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:26.632805 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:26.632805 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:26.632805 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:26.632805 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:26.632805 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:26.633752 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:26.633752 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:26.633752 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:26.633752 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:26.633752 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:26.633752 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:26.633752 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:26.633752 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:26.633752 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:26.633752 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:26.633752 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:26.633752 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:26.633752 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:26.633752 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:26.633752 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:26.633752 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:26.633752 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:26.634653 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:26.634653 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:26.634653 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:26.634653 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:26.634653 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 278/561] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:26.635268 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7332bdd-5f9d-4707-8d24-d41bf26b5022 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:26.635540 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7332bdd-5f9d-4707-8d24-d41bf26b5022 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:26.635819 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7332bdd-5f9d-4707-8d24-d41bf26b5022 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:26.641109 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 279/562] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:26.641109 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7332bdd-5f9d-4707-8d24-d41bf26b5022 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:26.670337 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:26.695030 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabe7a39-d004-4946-a41b-262fe7462a6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:26.696508 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabe7a39-d004-4946-a41b-262fe7462a6c None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:26.743104 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7332bdd-5f9d-4707-8d24-d41bf26b5022 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:26.776290 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e68a17-6fa4-45df-960c-d3b241845f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:26.776959 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e68a17-6fa4-45df-960c-d3b241845f98 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:26.777245 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e68a17-6fa4-45df-960c-d3b241845f98 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:26.777523 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e68a17-6fa4-45df-960c-d3b241845f98 None admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:26.779931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e68a17-6fa4-45df-960c-d3b241845f98 None admin] RBAC: Adding query filter params (name=service) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:26.779931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e68a17-6fa4-45df-960c-d3b241845f98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:26.779931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e68a17-6fa4-45df-960c-d3b241845f98 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:26.788737 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 280/563] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 21 22:35:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:26.789831 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e68a17-6fa4-45df-960c-d3b241845f98 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:26.795377 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 284/564] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:35:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:35:26.796642 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659541e-384d-423f-939e-691891b4670c None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:26.796933 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659541e-384d-423f-939e-691891b4670c None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:26.797205 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659541e-384d-423f-939e-691891b4670c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:26.805277 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:26.815987 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-7659541e-384d-423f-939e-691891b4670c None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:26.817359 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-7659541e-384d-423f-939e-691891b4670c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:26.885494 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:26.886237 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 285/565] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:35:26.887807 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91edb67-7365-451a-b14c-0f5fce972976 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:26.888353 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91edb67-7365-451a-b14c-0f5fce972976 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:26.888863 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91edb67-7365-451a-b14c-0f5fce972976 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:26.890676 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 286/566] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:26.891446 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91edb67-7365-451a-b14c-0f5fce972976 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:26.895653 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e68a17-6fa4-45df-960c-d3b241845f98 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:26.932016 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae054657-e03c-48b3-9218-7e297aa343e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:26.932796 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae054657-e03c-48b3-9218-7e297aa343e7 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:26.933085 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae054657-e03c-48b3-9218-7e297aa343e7 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:26.933362 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae054657-e03c-48b3-9218-7e297aa343e7 None admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:26.934078 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae054657-e03c-48b3-9218-7e297aa343e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:26.934383 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae054657-e03c-48b3-9218-7e297aa343e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:26.935785 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae054657-e03c-48b3-9218-7e297aa343e7 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:26.947573 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 281/567] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:26.948455 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae054657-e03c-48b3-9218-7e297aa343e7 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:26.999807 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91edb67-7365-451a-b14c-0f5fce972976 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:27.034761 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db689e30-1973-428f-a9be-c5c071a8068c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:27.035653 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db689e30-1973-428f-a9be-c5c071a8068c None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:27.035941 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db689e30-1973-428f-a9be-c5c071a8068c None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:27.036216 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db689e30-1973-428f-a9be-c5c071a8068c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:27.038813 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:27.045835 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae054657-e03c-48b3-9218-7e297aa343e7 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:27.047365 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db689e30-1973-428f-a9be-c5c071a8068c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:27.051988 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db689e30-1973-428f-a9be-c5c071a8068c None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:27.065671 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-db689e30-1973-428f-a9be-c5c071a8068c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:27.065671 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:27.065671 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:27.065671 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:27.065671 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:27.065671 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:27.065671 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:27.065671 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:27.065671 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:27.065671 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:27.065671 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:27.065671 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:27.065671 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:27.065671 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:27.065671 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:27.066569 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:27.066569 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:27.066569 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:27.066569 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:27.066569 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:27.066569 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:27.066569 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:27.066569 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:27.066569 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:27.066569 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:27.066569 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:27.066569 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:27.066569 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:27.066569 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:27.066569 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:27.066569 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:27.066569 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:27.067762 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:27.067762 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:27.067762 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:27.067762 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:27.067762 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 287/568] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:27.068042 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9143caa-68a3-4416-9c47-31c5de0c3710 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:27.068267 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9143caa-68a3-4416-9c47-31c5de0c3710 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:27.068533 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9143caa-68a3-4416-9c47-31c5de0c3710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:27.083149 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dc83f3-2e75-4ee4-b91a-efd4fa8b234a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:27.083946 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc83f3-2e75-4ee4-b91a-efd4fa8b234a None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:27.084230 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc83f3-2e75-4ee4-b91a-efd4fa8b234a None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:27.084365 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-a9143caa-68a3-4416-9c47-31c5de0c3710 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:27.084497 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc83f3-2e75-4ee4-b91a-efd4fa8b234a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:27.085984 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-a9143caa-68a3-4416-9c47-31c5de0c3710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:27.095978 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dc83f3-2e75-4ee4-b91a-efd4fa8b234a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:27.097432 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dc83f3-2e75-4ee4-b91a-efd4fa8b234a None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:27.116417 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-00dc83f3-2e75-4ee4-b91a-efd4fa8b234a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:27.116417 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:27.116417 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:27.116417 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:27.116417 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:27.116417 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:27.116417 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:27.116417 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:27.116417 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:27.116417 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:27.116417 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:27.116417 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:27.116417 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:27.116417 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:27.116417 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:27.117291 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:27.117291 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:27.117291 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:27.117291 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:27.117291 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:27.117291 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:27.117291 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:27.117291 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:27.117291 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:27.117291 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:27.117291 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:27.117291 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:27.117291 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:27.117291 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:27.117291 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:27.117291 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:27.117291 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:27.118184 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:27.118184 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:27.118184 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:27.118184 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:27.118184 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 282/569] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:27.118478 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dc83f3-2e75-4ee4-b91a-efd4fa8b234a None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:27.121705 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:27.154851 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 288/570] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:35:27.155694 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9143caa-68a3-4416-9c47-31c5de0c3710 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:27.226322 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dc83f3-2e75-4ee4-b91a-efd4fa8b234a None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:27.254501 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9143caa-68a3-4416-9c47-31c5de0c3710 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:27.259690 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b960e-e42a-4f59-b0c4-13f551322680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:27.260496 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b960e-e42a-4f59-b0c4-13f551322680 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:27.260763 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b960e-e42a-4f59-b0c4-13f551322680 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:27.261036 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b960e-e42a-4f59-b0c4-13f551322680 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:27.270600 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:27.271148 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235b960e-e42a-4f59-b0c4-13f551322680 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:35:27.279479 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-235b960e-e42a-4f59-b0c4-13f551322680 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:35:27.279479 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:27.279479 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:27.279479 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:27.279479 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:27.279479 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:27.279479 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:27.279479 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:27.279479 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:27.279479 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:27.279479 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:27.279479 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:27.279479 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:35:27.279479 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:35:27.279479 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:35:27.279479 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:35:27.286036 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:35:27.286036 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:35:27.286036 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:35:27.286036 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:35:27.286036 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:27.286036 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:27.286036 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:27.286036 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:27.286036 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:27.286036 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:27.286036 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:27.286036 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:27.286036 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:27.286036 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:27.286036 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:27.286036 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:27.286931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:27.286931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:27.286931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:27.286931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:27.286931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:35:27.286931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:35:27.286931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:35:27.286931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:35:27.286931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:35:27.286931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:35:27.286931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:35:27.286931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:27.286931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 283/571] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 21 22:35:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:27.286931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b960e-e42a-4f59-b0c4-13f551322680 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:27.296615 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c1a4fd-041f-4f6f-8150-f103454fcdb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:27.297415 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1a4fd-041f-4f6f-8150-f103454fcdb4 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:27.297701 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1a4fd-041f-4f6f-8150-f103454fcdb4 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:27.297984 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1a4fd-041f-4f6f-8150-f103454fcdb4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:27.305146 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c1a4fd-041f-4f6f-8150-f103454fcdb4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:27.306577 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c1a4fd-041f-4f6f-8150-f103454fcdb4 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:27.320576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-e0c1a4fd-041f-4f6f-8150-f103454fcdb4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:35:27.320576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:27.320576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:27.320576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:27.320576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:27.320576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:27.320576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:27.320576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:27.320576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:27.320576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:27.320576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:27.320576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:27.320576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:35:27.320576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:35:27.320576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:35:27.321645 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:35:27.321645 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:27.321645 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:27.321645 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:27.321645 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:27.321645 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:27.321645 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:27.321645 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:27.321645 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:27.321645 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:27.321645 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:27.321645 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:27.321645 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:27.321645 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:27.321645 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:27.321645 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:27.321645 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:27.322761 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:35:27.322761 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:35:27.322761 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:35:27.322761 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:35:27.322761 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:35:27.322761 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:35:27.322761 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:35:27.322761 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:27.322761 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 289/572] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:35:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:27.322761 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c1a4fd-041f-4f6f-8150-f103454fcdb4 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:27.346310 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:27.384030 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b960e-e42a-4f59-b0c4-13f551322680 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:27.416993 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6974a44-02ff-46cc-9206-7f7dbd758198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:27.417649 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6974a44-02ff-46cc-9206-7f7dbd758198 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:27.417933 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6974a44-02ff-46cc-9206-7f7dbd758198 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:27.418211 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6974a44-02ff-46cc-9206-7f7dbd758198 None admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:27.418975 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6974a44-02ff-46cc-9206-7f7dbd758198 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:27.419280 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6974a44-02ff-46cc-9206-7f7dbd758198 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:27.420652 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6974a44-02ff-46cc-9206-7f7dbd758198 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:27.429042 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c1a4fd-041f-4f6f-8150-f103454fcdb4 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:27.433052 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 284/573] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:27.433842 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6974a44-02ff-46cc-9206-7f7dbd758198 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:27.466073 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e42a441-9998-46bb-819e-734f4b205200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:27.466728 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42a441-9998-46bb-819e-734f4b205200 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:27.467035 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42a441-9998-46bb-819e-734f4b205200 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:27.467303 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42a441-9998-46bb-819e-734f4b205200 None admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:27.468019 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e42a441-9998-46bb-819e-734f4b205200 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:27.468322 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e42a441-9998-46bb-819e-734f4b205200 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:27.469678 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e42a441-9998-46bb-819e-734f4b205200 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:27.483809 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 290/574] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:27.484638 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e42a441-9998-46bb-819e-734f4b205200 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:27.518464 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:27.536646 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6974a44-02ff-46cc-9206-7f7dbd758198 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:27.559467 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:27.570594 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6747d65b-37eb-4044-9fd8-4f8ebcb7068a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:27.571409 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747d65b-37eb-4044-9fd8-4f8ebcb7068a None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:27.571686 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747d65b-37eb-4044-9fd8-4f8ebcb7068a None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:27.571955 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747d65b-37eb-4044-9fd8-4f8ebcb7068a None admin] PATH_INFO: `/v3/roles/service` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:27.581755 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6747d65b-37eb-4044-9fd8-4f8ebcb7068a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:35:27.590207 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-6747d65b-37eb-4044-9fd8-4f8ebcb7068a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:35:27.590207 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:27.590207 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:27.590207 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:27.590207 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:27.590207 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:27.590207 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:27.590207 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:27.590207 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:27.590207 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:27.590207 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:27.590207 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:27.590207 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:35:27.590207 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:35:27.590207 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:35:27.590207 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:35:27.591259 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:35:27.591259 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:35:27.591259 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:35:27.591259 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:35:27.591259 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:27.591259 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:27.591259 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:27.591259 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:27.591259 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:27.591259 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:27.591259 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:27.591259 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:27.591259 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:27.591259 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:27.591259 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:27.591259 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:27.592090 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:27.592090 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:27.592090 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:27.592090 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:27.592090 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:35:27.592090 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:35:27.592090 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:35:27.592090 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:35:27.592090 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:35:27.592090 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:35:27.592090 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:35:27.592090 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:27.593350 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 285/575] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 21 22:35:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:27.594376 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6747d65b-37eb-4044-9fd8-4f8ebcb7068a None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:27.616329 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e42a441-9998-46bb-819e-734f4b205200 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:27.650959 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beaba01-07d2-4c4b-b172-e1e330331758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:27.651617 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beaba01-07d2-4c4b-b172-e1e330331758 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:27.651901 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beaba01-07d2-4c4b-b172-e1e330331758 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:27.652176 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beaba01-07d2-4c4b-b172-e1e330331758 None admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:27.652914 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beaba01-07d2-4c4b-b172-e1e330331758 None admin] RBAC: Adding query filter params (name=service) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:27.653216 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beaba01-07d2-4c4b-b172-e1e330331758 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:27.654564 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beaba01-07d2-4c4b-b172-e1e330331758 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:27.662468 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 291/576] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 21 22:35:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:27.663140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beaba01-07d2-4c4b-b172-e1e330331758 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:27.695824 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6747d65b-37eb-4044-9fd8-4f8ebcb7068a None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:27.728610 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee5898c-2da8-4212-81ae-77da2d460801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:27.729273 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee5898c-2da8-4212-81ae-77da2d460801 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:27.729567 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee5898c-2da8-4212-81ae-77da2d460801 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:27.729848 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee5898c-2da8-4212-81ae-77da2d460801 None admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:27.730634 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee5898c-2da8-4212-81ae-77da2d460801 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:27.730973 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee5898c-2da8-4212-81ae-77da2d460801 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:27.732396 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee5898c-2da8-4212-81ae-77da2d460801 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:27.744740 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 286/577] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 22:35:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:27.745669 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee5898c-2da8-4212-81ae-77da2d460801 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:27.762442 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:27.766175 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beaba01-07d2-4c4b-b172-e1e330331758 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:27.790490 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:27.799114 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4b07f0-38b5-4f31-8569-7c6d74fec29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:27.799901 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b07f0-38b5-4f31-8569-7c6d74fec29f None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:27.800182 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b07f0-38b5-4f31-8569-7c6d74fec29f None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:27.800456 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b07f0-38b5-4f31-8569-7c6d74fec29f None admin] PATH_INFO: `/v3/users/nova` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:27.808981 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4b07f0-38b5-4f31-8569-7c6d74fec29f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:27.811148 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4b07f0-38b5-4f31-8569-7c6d74fec29f None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:27.827028 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-cd4b07f0-38b5-4f31-8569-7c6d74fec29f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:35:27.827028 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:27.827028 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:27.827028 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:27.827028 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:27.827028 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:27.827028 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:27.827028 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:27.827028 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:27.827028 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:27.827028 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:27.827028 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:27.827028 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:35:27.827028 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:35:27.827028 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:35:27.827028 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:35:27.828102 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:27.828102 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:27.828102 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:35:27.828102 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:35:27.828102 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:35:27.828102 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:35:27.828102 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:27.828102 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:27.828102 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:27.828102 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:27.828102 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:27.828102 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:27.828102 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:27.828102 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:27.828102 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:27.828102 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:27.828102 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:27.828102 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:27.829053 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:27.829053 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:27.829053 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:35:27.829053 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:35:27.829053 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:35:27.829053 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:35:27.829053 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:35:27.829053 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:35:27.829053 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:35:27.829053 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:27.829053 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 292/578] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Jul 21 22:35:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:27.829053 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4b07f0-38b5-4f31-8569-7c6d74fec29f None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:27.851856 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee5898c-2da8-4212-81ae-77da2d460801 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:27.886179 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd8d82-4120-4f46-bac2-d9f0ff068b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:27.886984 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd8d82-4120-4f46-bac2-d9f0ff068b8a None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:27.887260 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd8d82-4120-4f46-bac2-d9f0ff068b8a None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:27.887539 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd8d82-4120-4f46-bac2-d9f0ff068b8a None admin] PATH_INFO: `/v3/projects/service` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:27.893997 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd8d82-4120-4f46-bac2-d9f0ff068b8a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:27.895435 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd8d82-4120-4f46-bac2-d9f0ff068b8a None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:27.908707 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-91fd8d82-4120-4f46-bac2-d9f0ff068b8a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:35:27.908707 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:27.908707 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:27.908707 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:27.908707 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:27.908707 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:27.908707 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:27.908707 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:27.908707 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:27.908707 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:27.908707 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:27.908707 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:27.908707 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:35:27.908707 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:35:27.908707 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:35:27.909739 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:35:27.909739 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:27.909739 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:27.909739 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:27.909739 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:27.909739 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:27.909739 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:27.909739 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:27.909739 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:27.909739 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:27.909739 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:27.909739 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:27.909739 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:27.909739 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:27.909739 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:27.909739 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:27.909739 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:27.910847 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:35:27.910847 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:35:27.910847 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:35:27.910847 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:35:27.910847 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:35:27.910847 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:35:27.910847 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:35:27.910847 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:27.910847 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 287/579] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:35:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:27.910847 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd8d82-4120-4f46-bac2-d9f0ff068b8a None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:27.930188 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4b07f0-38b5-4f31-8569-7c6d74fec29f None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:27.963257 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb4cf44-654f-45a9-920c-416147c3b682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:27.964006 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb4cf44-654f-45a9-920c-416147c3b682 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:27.964297 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb4cf44-654f-45a9-920c-416147c3b682 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:27.964577 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb4cf44-654f-45a9-920c-416147c3b682 None admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:27.965324 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb4cf44-654f-45a9-920c-416147c3b682 None admin] RBAC: Adding query filter params (name=service) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:27.965632 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb4cf44-654f-45a9-920c-416147c3b682 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:27.967014 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb4cf44-654f-45a9-920c-416147c3b682 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:27.975023 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 293/580] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 21 22:35:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:27.975794 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb4cf44-654f-45a9-920c-416147c3b682 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:27.997154 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:28.014680 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd8d82-4120-4f46-bac2-d9f0ff068b8a None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:28.017050 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:28.047684 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575e4f83-5d14-4575-b7e2-4cccb8a5b697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:28.048409 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e4f83-5d14-4575-b7e2-4cccb8a5b697 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:28.048691 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e4f83-5d14-4575-b7e2-4cccb8a5b697 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:28.048968 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e4f83-5d14-4575-b7e2-4cccb8a5b697 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:28.059965 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575e4f83-5d14-4575-b7e2-4cccb8a5b697 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:28.061392 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575e4f83-5d14-4575-b7e2-4cccb8a5b697 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:28.073938 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb4cf44-654f-45a9-920c-416147c3b682 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:28.078859 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-575e4f83-5d14-4575-b7e2-4cccb8a5b697 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:28.078859 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:28.078859 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:28.078859 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:28.078859 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:28.078859 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:28.078859 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:28.078859 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:28.078859 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:28.078859 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:28.078859 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:28.078859 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:28.078859 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:28.078859 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:28.078859 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:28.079882 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:28.079882 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:28.079882 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:28.079882 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:28.079882 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:28.079882 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:28.079882 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:28.079882 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:28.079882 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:28.079882 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:28.079882 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:28.079882 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:28.079882 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:28.079882 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:28.079882 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:28.079882 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:28.079882 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:28.080754 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:28.080754 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:28.080754 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:28.080754 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:28.080754 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 288/581] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:28.081229 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575e4f83-5d14-4575-b7e2-4cccb8a5b697 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:28.109042 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b886be-5776-43b9-a710-549a40a9f1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:28.109929 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b886be-5776-43b9-a710-549a40a9f1d9 None admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:28.110212 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b886be-5776-43b9-a710-549a40a9f1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:28.110490 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b886be-5776-43b9-a710-549a40a9f1d9 None admin] PATH_INFO: `/v3/projects/aa11ab8229b642ce962193b74001d3bc/users/927778d005fd48a4b4b8fd3c4419bb36/roles/715d866890d14923b11470d544b4f960` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:28.125034 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b886be-5776-43b9-a710-549a40a9f1d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa11ab8229b642ce962193b74001d3bc, user_id=927778d005fd48a4b4b8fd3c4419bb36, role_id=715d866890d14923b11470d544b4f960)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:28.126467 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b886be-5776-43b9-a710-549a40a9f1d9 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:28.180147 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575e4f83-5d14-4575-b7e2-4cccb8a5b697 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:28.212135 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0508ea88-3a5b-45f7-8585-1c6a84023230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:28.212787 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0508ea88-3a5b-45f7-8585-1c6a84023230 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:28.213066 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0508ea88-3a5b-45f7-8585-1c6a84023230 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:28.213335 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0508ea88-3a5b-45f7-8585-1c6a84023230 None admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:28.214083 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 294/582] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 21 22:35:27 2022] PUT /identity/v3/projects/aa11ab8229b642ce962193b74001d3bc/users/927778d005fd48a4b4b8fd3c4419bb36/roles/715d866890d14923b11470d544b4f960 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:28.214185 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0508ea88-3a5b-45f7-8585-1c6a84023230 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:28.214447 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0508ea88-3a5b-45f7-8585-1c6a84023230 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:28.214928 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b886be-5776-43b9-a710-549a40a9f1d9 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:28.215892 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0508ea88-3a5b-45f7-8585-1c6a84023230 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:28.242617 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 289/583] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Jul 21 22:35:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:28.242617 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0508ea88-3a5b-45f7-8585-1c6a84023230 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:28.258362 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:28.314115 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:28.346253 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b886be-5776-43b9-a710-549a40a9f1d9 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:28.353948 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0508ea88-3a5b-45f7-8585-1c6a84023230 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:28.386865 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d100c-3997-4583-812c-99d0d6287854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:28.387667 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d100c-3997-4583-812c-99d0d6287854 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:28.388035 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d100c-3997-4583-812c-99d0d6287854 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:28.388299 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d100c-3997-4583-812c-99d0d6287854 None admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:28.389559 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167d100c-3997-4583-812c-99d0d6287854 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:28.389943 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167d100c-3997-4583-812c-99d0d6287854 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:28.391484 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167d100c-3997-4583-812c-99d0d6287854 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:28.395933 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f02a1b2-8a74-4877-80a8-5fdde205dad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:28.396731 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f02a1b2-8a74-4877-80a8-5fdde205dad7 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:28.397015 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f02a1b2-8a74-4877-80a8-5fdde205dad7 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:28.397291 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f02a1b2-8a74-4877-80a8-5fdde205dad7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:28.406820 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 295/584] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 22:35:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:28.406820 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d100c-3997-4583-812c-99d0d6287854 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:28.408415 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f02a1b2-8a74-4877-80a8-5fdde205dad7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:28.409850 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f02a1b2-8a74-4877-80a8-5fdde205dad7 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:28.427555 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-7f02a1b2-8a74-4877-80a8-5fdde205dad7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:28.427555 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:28.427555 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:28.427555 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:28.427555 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:28.427555 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:28.427555 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:28.427555 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:28.427555 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:28.427555 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:28.427555 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:28.427555 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:28.427555 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:28.427555 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:28.427555 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:28.429190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:28.429190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:28.429190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:28.429190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:28.429190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:28.429190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:28.429190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:28.429190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:28.429190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:28.429190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:28.429190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:28.429190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:28.429190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:28.429190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:28.429190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:28.429190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:28.429190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:28.430516 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:28.430516 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:28.430516 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:28.430516 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:28.430516 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 290/585] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:28.430516 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f02a1b2-8a74-4877-80a8-5fdde205dad7 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:28.509602 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:28.526323 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d100c-3997-4583-812c-99d0d6287854 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:28.545880 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f02a1b2-8a74-4877-80a8-5fdde205dad7 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:28.557150 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:28.563147 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2cf7d5-7190-41dd-8a7c-868a6a5b4843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:28.563925 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2cf7d5-7190-41dd-8a7c-868a6a5b4843 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:28.564257 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2cf7d5-7190-41dd-8a7c-868a6a5b4843 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:28.564484 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2cf7d5-7190-41dd-8a7c-868a6a5b4843 None admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:28.565279 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2cf7d5-7190-41dd-8a7c-868a6a5b4843 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:28.565589 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2cf7d5-7190-41dd-8a7c-868a6a5b4843 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:28.567002 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2cf7d5-7190-41dd-8a7c-868a6a5b4843 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:28.579568 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 296/586] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:28.580186 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2cf7d5-7190-41dd-8a7c-868a6a5b4843 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:28.589296 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9edee9-a92a-424c-8fd3-9b91118d1a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:28.590097 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9edee9-a92a-424c-8fd3-9b91118d1a30 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:28.590381 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9edee9-a92a-424c-8fd3-9b91118d1a30 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:28.590659 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9edee9-a92a-424c-8fd3-9b91118d1a30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:28.601342 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9edee9-a92a-424c-8fd3-9b91118d1a30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:28.603065 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9edee9-a92a-424c-8fd3-9b91118d1a30 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:28.621806 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-1c9edee9-a92a-424c-8fd3-9b91118d1a30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:28.621806 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:28.621806 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:28.621806 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:28.621806 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:28.621806 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:28.621806 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:28.621806 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:28.621806 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:28.621806 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:28.621806 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:28.621806 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:28.621806 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:28.621806 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:28.621806 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:28.622931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:28.622931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:28.622931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:28.622931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:28.622931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:28.622931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:28.622931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:28.622931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:28.622931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:28.622931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:28.622931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:28.622931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:28.622931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:28.622931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:28.622931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:28.622931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:28.622931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:28.623815 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:28.623815 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:28.623815 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:28.623815 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:28.623815 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 291/587] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:28.623815 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9edee9-a92a-424c-8fd3-9b91118d1a30 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:28.733741 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9edee9-a92a-424c-8fd3-9b91118d1a30 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:28.739950 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2cf7d5-7190-41dd-8a7c-868a6a5b4843 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:28.754879 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:28.772970 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5882cac0-ded0-488d-8ffb-27e226cc8e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:28.773604 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882cac0-ded0-488d-8ffb-27e226cc8e59 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:28.773890 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882cac0-ded0-488d-8ffb-27e226cc8e59 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:28.774167 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882cac0-ded0-488d-8ffb-27e226cc8e59 None admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:28.774919 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5882cac0-ded0-488d-8ffb-27e226cc8e59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:28.775236 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5882cac0-ded0-488d-8ffb-27e226cc8e59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:28.776371 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2361a3c4-1320-46dd-92bb-ef44e4f32c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:28.776602 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5882cac0-ded0-488d-8ffb-27e226cc8e59 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:28.777027 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361a3c4-1320-46dd-92bb-ef44e4f32c56 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:28.777313 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361a3c4-1320-46dd-92bb-ef44e4f32c56 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:28.777607 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361a3c4-1320-46dd-92bb-ef44e4f32c56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:28.778428 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2361a3c4-1320-46dd-92bb-ef44e4f32c56 None admin] RBAC: Adding query filter params (role.id=715d866890d14923b11470d544b4f960, scope.project.id=aa11ab8229b642ce962193b74001d3bc, user.id=ba7eef4e1b27476ab4016449d9c3a4b9) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:28.778751 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2361a3c4-1320-46dd-92bb-ef44e4f32c56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:28.780207 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2361a3c4-1320-46dd-92bb-ef44e4f32c56 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:28.786103 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 297/588] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 22:35:28 2022] GET /identity/v3/role_assignments?user.id=ba7eef4e1b27476ab4016449d9c3a4b9&scope.project.id=aa11ab8229b642ce962193b74001d3bc&role.id=715d866890d14923b11470d544b4f960 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:28.786103 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2361a3c4-1320-46dd-92bb-ef44e4f32c56 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:28.798259 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 292/589] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:28.799316 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5882cac0-ded0-488d-8ffb-27e226cc8e59 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:28.876854 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:28.920511 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5882cac0-ded0-488d-8ffb-27e226cc8e59 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:28.941614 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2361a3c4-1320-46dd-92bb-ef44e4f32c56 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:28.960809 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716fd27c-a0e5-46fa-a696-e6cfb5f0fa61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:28.961635 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716fd27c-a0e5-46fa-a696-e6cfb5f0fa61 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:28.962019 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716fd27c-a0e5-46fa-a696-e6cfb5f0fa61 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:28.962444 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716fd27c-a0e5-46fa-a696-e6cfb5f0fa61 None admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:28.963331 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716fd27c-a0e5-46fa-a696-e6cfb5f0fa61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:28.964350 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716fd27c-a0e5-46fa-a696-e6cfb5f0fa61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:28.966073 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716fd27c-a0e5-46fa-a696-e6cfb5f0fa61 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:28.980980 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 293/590] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:28.981909 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716fd27c-a0e5-46fa-a696-e6cfb5f0fa61 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:28.992892 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41768316-281c-4c8b-85f8-3689bc88b44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:28.993904 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41768316-281c-4c8b-85f8-3689bc88b44c None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:28.994360 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41768316-281c-4c8b-85f8-3689bc88b44c None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:28.994818 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41768316-281c-4c8b-85f8-3689bc88b44c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:29.005753 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41768316-281c-4c8b-85f8-3689bc88b44c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:29.007844 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41768316-281c-4c8b-85f8-3689bc88b44c None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:29.027347 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-41768316-281c-4c8b-85f8-3689bc88b44c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 22:35:29.027347 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:29.027347 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:29.027347 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:29.027347 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:29.027347 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:29.027347 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:29.027347 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:29.027347 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:29.027347 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:29.027347 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:29.027347 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:29.027347 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:35:29.027347 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:35:29.027347 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:35:29.027347 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:35:29.028518 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:29.028518 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:29.028518 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:35:29.028518 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:35:29.028518 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:35:29.028518 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:35:29.028518 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:29.028518 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:29.028518 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:29.028518 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:29.028518 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:29.028518 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:29.028518 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:29.028518 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:29.028518 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:29.028518 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:29.028518 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:29.028518 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:29.029657 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:29.029657 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:29.029657 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:35:29.029657 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:35:29.029657 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:35:29.029657 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:35:29.029657 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:35:29.029657 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:35:29.029657 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 22:35:29.029657 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:29.031115 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:29.033166 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 298/591] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 21 22:35:28 2022] GET /identity/v3/users/neutron => generated 85 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:29.034497 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41768316-281c-4c8b-85f8-3689bc88b44c None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:29.103165 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716fd27c-a0e5-46fa-a696-e6cfb5f0fa61 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:29.141837 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119c69e4-59d7-443e-aecd-5d4170a1630b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:29.142714 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c69e4-59d7-443e-aecd-5d4170a1630b None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:29.143134 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c69e4-59d7-443e-aecd-5d4170a1630b None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:29.143505 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c69e4-59d7-443e-aecd-5d4170a1630b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:29.153086 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119c69e4-59d7-443e-aecd-5d4170a1630b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:29.154883 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119c69e4-59d7-443e-aecd-5d4170a1630b None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:29.168384 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41768316-281c-4c8b-85f8-3689bc88b44c None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:29.171505 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-119c69e4-59d7-443e-aecd-5d4170a1630b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 22:35:29.171505 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:29.171505 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:29.171505 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:29.171505 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:29.171505 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:29.171505 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:29.171505 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:29.171505 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:29.171505 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:29.171505 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:29.171505 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:29.171505 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:35:29.171505 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:35:29.171505 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:35:29.171505 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:35:29.172628 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:29.172628 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:29.172628 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:35:29.172628 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:35:29.172628 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:35:29.172628 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:35:29.172628 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:29.172628 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:29.172628 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:29.172628 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:29.172628 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:29.172628 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:29.172628 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:29.172628 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:29.172628 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:29.172628 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:29.172628 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:29.172628 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:29.173858 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:29.173858 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:29.173858 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:35:29.173858 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:35:29.173858 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:35:29.173858 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:35:29.173858 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:35:29.173858 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:35:29.173858 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 22:35:29.173858 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:29.180646 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 294/592] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:35:28 2022] GET /identity/v3/users/cinder => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:29.180646 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119c69e4-59d7-443e-aecd-5d4170a1630b None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:29.195303 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:29.222440 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdd7cbc-eaee-4f7b-8dc9-7ec63f741547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:29.223274 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd7cbc-eaee-4f7b-8dc9-7ec63f741547 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:29.223567 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd7cbc-eaee-4f7b-8dc9-7ec63f741547 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:29.223846 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd7cbc-eaee-4f7b-8dc9-7ec63f741547 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:29.233677 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdd7cbc-eaee-4f7b-8dc9-7ec63f741547 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:29.236440 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdd7cbc-eaee-4f7b-8dc9-7ec63f741547 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:29.256213 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-fbdd7cbc-eaee-4f7b-8dc9-7ec63f741547 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:35:29.256213 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:29.256213 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:29.256213 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:29.256213 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:29.256213 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:29.256213 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:29.256213 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:29.256213 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:29.256213 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:29.256213 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:29.256213 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:29.256213 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:35:29.256213 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:35:29.256213 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:35:29.257162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:35:29.257162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:29.257162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:29.257162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:29.257162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:29.257162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:29.257162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:29.257162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:29.257162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:29.257162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:29.257162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:29.257162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:29.257162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:29.257162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:29.257162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:29.257162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:29.257162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:29.258634 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:35:29.258634 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:35:29.258634 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:35:29.258634 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:35:29.258634 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:35:29.258634 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:35:29.258634 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:35:29.258634 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:29.258634 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 299/593] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:35:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:29.258634 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdd7cbc-eaee-4f7b-8dc9-7ec63f741547 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:29.303105 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:29.307979 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119c69e4-59d7-443e-aecd-5d4170a1630b None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:29.361238 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5538cbb-cf95-44db-9604-ddf6ba7c234a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:29.362276 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5538cbb-cf95-44db-9604-ddf6ba7c234a None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:29.362891 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5538cbb-cf95-44db-9604-ddf6ba7c234a None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:29.363374 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5538cbb-cf95-44db-9604-ddf6ba7c234a None admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:29.364961 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5538cbb-cf95-44db-9604-ddf6ba7c234a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:29.365494 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5538cbb-cf95-44db-9604-ddf6ba7c234a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:29.368070 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5538cbb-cf95-44db-9604-ddf6ba7c234a None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:29.392664 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 295/594] 10.222.0.52 () {56 vars in 1139 bytes} [Thu Jul 21 22:35:29 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:29.393936 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5538cbb-cf95-44db-9604-ddf6ba7c234a None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:29.394548 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdd7cbc-eaee-4f7b-8dc9-7ec63f741547 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:29.442129 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679420b0-1832-40c2-8d4d-717d102b0e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:29.442824 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679420b0-1832-40c2-8d4d-717d102b0e29 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:29.443111 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679420b0-1832-40c2-8d4d-717d102b0e29 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:29.443453 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679420b0-1832-40c2-8d4d-717d102b0e29 None admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:29.444317 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679420b0-1832-40c2-8d4d-717d102b0e29 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:29.444626 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679420b0-1832-40c2-8d4d-717d102b0e29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:29.447300 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679420b0-1832-40c2-8d4d-717d102b0e29 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:29.469347 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 300/595] 10.222.0.52 () {56 vars in 1137 bytes} [Thu Jul 21 22:35:29 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:29.470169 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679420b0-1832-40c2-8d4d-717d102b0e29 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:29.523305 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:29.536423 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5538cbb-cf95-44db-9604-ddf6ba7c234a None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:29.573789 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c2e966-8f84-4418-9982-fcef7524d3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:29.574513 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2e966-8f84-4418-9982-fcef7524d3b8 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:29.574796 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2e966-8f84-4418-9982-fcef7524d3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:29.575073 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2e966-8f84-4418-9982-fcef7524d3b8 None admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:29.575912 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c2e966-8f84-4418-9982-fcef7524d3b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:29.576217 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c2e966-8f84-4418-9982-fcef7524d3b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:29.577890 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c2e966-8f84-4418-9982-fcef7524d3b8 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:29.587282 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679420b0-1832-40c2-8d4d-717d102b0e29 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:29.590996 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 296/596] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 22:35:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:29.591814 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c2e966-8f84-4418-9982-fcef7524d3b8 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:29.612135 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:29.625444 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc58da3-a9db-406d-98d6-9605d568e9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:29.626423 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc58da3-a9db-406d-98d6-9605d568e9b8 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:29.626896 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc58da3-a9db-406d-98d6-9605d568e9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:29.627315 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc58da3-a9db-406d-98d6-9605d568e9b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:29.638297 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc58da3-a9db-406d-98d6-9605d568e9b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:29.640060 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc58da3-a9db-406d-98d6-9605d568e9b8 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:29.660007 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-9cc58da3-a9db-406d-98d6-9605d568e9b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:29.660007 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:29.660007 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:29.660007 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:29.660007 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:29.660007 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:29.660007 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:29.660007 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:29.660007 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:29.660007 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:29.660007 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:29.660007 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:29.660007 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:29.660007 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:29.660007 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:29.661050 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:29.661050 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:29.661050 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:29.661050 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:29.661050 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:29.661050 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:29.661050 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:29.661050 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:29.661050 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:29.661050 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:29.661050 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:29.661050 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:29.661050 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:29.661050 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:29.661050 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:29.661050 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:29.661050 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:29.662144 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:29.662144 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:29.662144 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:29.662144 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:29.664611 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 301/597] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:29.665649 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc58da3-a9db-406d-98d6-9605d568e9b8 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:29.697362 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c2e966-8f84-4418-9982-fcef7524d3b8 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:29.732669 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72905400-1a18-4fb3-b12d-20284fbb792f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:29.733446 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72905400-1a18-4fb3-b12d-20284fbb792f None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:29.733727 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72905400-1a18-4fb3-b12d-20284fbb792f None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:29.734045 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72905400-1a18-4fb3-b12d-20284fbb792f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:29.745686 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72905400-1a18-4fb3-b12d-20284fbb792f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:29.747135 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72905400-1a18-4fb3-b12d-20284fbb792f None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:29.764485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-72905400-1a18-4fb3-b12d-20284fbb792f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:29.764485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:29.764485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:29.764485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:29.764485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:29.764485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:29.764485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:29.764485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:29.764485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:29.764485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:29.764485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:29.764485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:29.764485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:29.764485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:29.764485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:29.765570 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:29.765570 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:29.765570 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:29.765570 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:29.765570 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:29.765570 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:29.765570 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:29.765570 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:29.765570 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:29.765570 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:29.765570 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:29.765570 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:29.765570 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:29.765570 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:29.765570 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:29.765570 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:29.765570 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:29.766650 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:29.766650 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:29.766650 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:29.766650 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:29.767117 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:29.770334 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 297/598] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:29.770975 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72905400-1a18-4fb3-b12d-20284fbb792f None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:29.777230 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc58da3-a9db-406d-98d6-9605d568e9b8 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:29.822806 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d32af05-4ea5-4a3a-9f5f-9153703306f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:29.822806 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d32af05-4ea5-4a3a-9f5f-9153703306f6 None admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:29.823132 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d32af05-4ea5-4a3a-9f5f-9153703306f6 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:29.823410 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d32af05-4ea5-4a3a-9f5f-9153703306f6 None admin] PATH_INFO: `/v3/projects/aa11ab8229b642ce962193b74001d3bc/users/6bfdb655a2a44b33ab724daaf73a294b/roles/715d866890d14923b11470d544b4f960` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:29.842709 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d32af05-4ea5-4a3a-9f5f-9153703306f6 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa11ab8229b642ce962193b74001d3bc, user_id=6bfdb655a2a44b33ab724daaf73a294b, role_id=715d866890d14923b11470d544b4f960)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:29.845060 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d32af05-4ea5-4a3a-9f5f-9153703306f6 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:29.860044 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:29.878329 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72905400-1a18-4fb3-b12d-20284fbb792f None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:29.911538 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0609141f-33e8-4b75-a225-5df7b371bbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:29.912163 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0609141f-33e8-4b75-a225-5df7b371bbf9 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:29.912447 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0609141f-33e8-4b75-a225-5df7b371bbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:29.912719 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0609141f-33e8-4b75-a225-5df7b371bbf9 None admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:29.914116 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0609141f-33e8-4b75-a225-5df7b371bbf9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:29.914419 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0609141f-33e8-4b75-a225-5df7b371bbf9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:29.915834 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0609141f-33e8-4b75-a225-5df7b371bbf9 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:29.929304 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 298/599] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:29.930025 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0609141f-33e8-4b75-a225-5df7b371bbf9 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:29.938645 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 302/600] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 21 22:35:29 2022] PUT /identity/v3/projects/aa11ab8229b642ce962193b74001d3bc/users/6bfdb655a2a44b33ab724daaf73a294b/roles/715d866890d14923b11470d544b4f960 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:29.940795 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d32af05-4ea5-4a3a-9f5f-9153703306f6 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:30.101332 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d32af05-4ea5-4a3a-9f5f-9153703306f6 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:30.132924 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0609141f-33e8-4b75-a225-5df7b371bbf9 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:30.146724 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8085ff-6b65-40c3-91ed-4f08fd0e517d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:30.148411 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8085ff-6b65-40c3-91ed-4f08fd0e517d None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:30.148838 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8085ff-6b65-40c3-91ed-4f08fd0e517d None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:30.149248 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8085ff-6b65-40c3-91ed-4f08fd0e517d None admin] PATH_INFO: `/v3/projects/service` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:30.153515 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:30.159630 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8085ff-6b65-40c3-91ed-4f08fd0e517d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:30.161356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8085ff-6b65-40c3-91ed-4f08fd0e517d None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:30.173196 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:30.174459 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4bce46-e81e-45f1-b868-962d078e8508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:30.175169 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4bce46-e81e-45f1-b868-962d078e8508 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:30.175451 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4bce46-e81e-45f1-b868-962d078e8508 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:30.175726 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4bce46-e81e-45f1-b868-962d078e8508 None admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:30.176450 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4bce46-e81e-45f1-b868-962d078e8508 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:30.176754 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4bce46-e81e-45f1-b868-962d078e8508 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:30.178111 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4bce46-e81e-45f1-b868-962d078e8508 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:30.184595 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-4c8085ff-6b65-40c3-91ed-4f08fd0e517d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:35:30.184595 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:30.184595 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:30.184595 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:30.184595 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:30.184595 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:30.184595 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:30.184595 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:30.184595 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:30.184595 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:30.184595 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:30.184595 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:30.184595 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:35:30.184595 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:35:30.184595 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:35:30.185662 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:35:30.185662 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:30.185662 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:30.185662 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:30.185662 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:30.185662 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:30.185662 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:30.185662 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:30.185662 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:30.185662 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:30.185662 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:30.185662 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:30.185662 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:30.185662 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:30.185662 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:30.185662 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:30.185662 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:30.186695 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:35:30.186695 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:35:30.186695 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:35:30.186695 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:35:30.186695 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:35:30.186695 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:35:30.186695 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:35:30.186695 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:30.188756 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 303/601] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:35:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:30.190817 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 299/602] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:30.195602 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8085ff-6b65-40c3-91ed-4f08fd0e517d None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:30.196112 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4bce46-e81e-45f1-b868-962d078e8508 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:30.313502 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4bce46-e81e-45f1-b868-962d078e8508 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:30.364482 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d67367-4e44-487e-ab58-ae94f8d4da79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:30.365573 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d67367-4e44-487e-ab58-ae94f8d4da79 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:30.365877 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d67367-4e44-487e-ab58-ae94f8d4da79 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:30.366204 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d67367-4e44-487e-ab58-ae94f8d4da79 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:30.386796 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d67367-4e44-487e-ab58-ae94f8d4da79 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:30.386796 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d67367-4e44-487e-ab58-ae94f8d4da79 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:30.417469 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-59d67367-4e44-487e-ab58-ae94f8d4da79 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:35:30.417469 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:30.417469 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:30.417469 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:30.417469 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:30.417469 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:30.417469 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:30.417469 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:30.417469 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:30.417469 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:30.417469 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:30.417469 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:30.417469 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:35:30.417469 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:35:30.417469 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:35:30.418418 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:35:30.418418 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:30.418418 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:30.418418 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:30.418418 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:30.418418 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:30.418418 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:30.418418 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:30.418418 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:30.418418 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:30.418418 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:30.418418 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:30.418418 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:30.418418 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:30.418418 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:30.418418 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:30.418418 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:30.419396 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:35:30.419396 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:35:30.419396 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:35:30.419396 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:35:30.419396 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:35:30.419396 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:35:30.419396 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:35:30.419396 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:30.419396 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 300/603] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:35:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:30.421323 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb1c6e7-cf16-4bc7-a7bb-e513be890dc6 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:30.421323 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb1c6e7-cf16-4bc7-a7bb-e513be890dc6 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:30.421323 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb1c6e7-cf16-4bc7-a7bb-e513be890dc6 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:30.426268 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 301/604] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:30.427136 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb1c6e7-cf16-4bc7-a7bb-e513be890dc6 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:30.439242 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8085ff-6b65-40c3-91ed-4f08fd0e517d None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:30.513868 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8805e3e3-1c04-49ec-8d33-10e05ee2f5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:30.515068 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8805e3e3-1c04-49ec-8d33-10e05ee2f5e8 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:30.515576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8805e3e3-1c04-49ec-8d33-10e05ee2f5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:30.516092 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8805e3e3-1c04-49ec-8d33-10e05ee2f5e8 None admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:30.517203 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8805e3e3-1c04-49ec-8d33-10e05ee2f5e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:30.517721 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8805e3e3-1c04-49ec-8d33-10e05ee2f5e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:30.521377 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8805e3e3-1c04-49ec-8d33-10e05ee2f5e8 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:30.524353 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:30.550661 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 304/605] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 22:35:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:30.553353 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ace0a-2f56-46bd-816b-4cd07fd648c4 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:30.555092 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ace0a-2f56-46bd-816b-4cd07fd648c4 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:30.556358 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ace0a-2f56-46bd-816b-4cd07fd648c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:30.591118 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-974ace0a-2f56-46bd-816b-4cd07fd648c4 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:30.593261 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-974ace0a-2f56-46bd-816b-4cd07fd648c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:30.630943 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb1c6e7-cf16-4bc7-a7bb-e513be890dc6 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:30.687414 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:30.701260 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9d76ce-94d0-4391-aa6d-c80abb87a067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:30.705322 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d76ce-94d0-4391-aa6d-c80abb87a067 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:30.705322 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d76ce-94d0-4391-aa6d-c80abb87a067 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:30.705322 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d76ce-94d0-4391-aa6d-c80abb87a067 None admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:30.705322 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9d76ce-94d0-4391-aa6d-c80abb87a067 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:30.705322 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9d76ce-94d0-4391-aa6d-c80abb87a067 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:30.706503 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9d76ce-94d0-4391-aa6d-c80abb87a067 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:30.721751 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 302/606] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 22:35:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:30.721751 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9d76ce-94d0-4391-aa6d-c80abb87a067 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:30.722136 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 305/607] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:35:30.736917 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974ace0a-2f56-46bd-816b-4cd07fd648c4 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:30.898878 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974ace0a-2f56-46bd-816b-4cd07fd648c4 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:30.941162 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9d76ce-94d0-4391-aa6d-c80abb87a067 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:30.952292 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:30.954269 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4d320c-3f89-4851-a6dd-37f001b07974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:30.955489 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4d320c-3f89-4851-a6dd-37f001b07974 None admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:30.955983 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4d320c-3f89-4851-a6dd-37f001b07974 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:30.956437 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4d320c-3f89-4851-a6dd-37f001b07974 None admin] PATH_INFO: `/v3/projects/aa11ab8229b642ce962193b74001d3bc/users/150f42651f754fda83f25fe3b408bce9/roles/715d866890d14923b11470d544b4f960` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:30.971893 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4d320c-3f89-4851-a6dd-37f001b07974 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa11ab8229b642ce962193b74001d3bc, user_id=150f42651f754fda83f25fe3b408bce9, role_id=715d866890d14923b11470d544b4f960)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:30.973649 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4d320c-3f89-4851-a6dd-37f001b07974 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:30.980704 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2a1291-7ccf-441c-b57d-71913d9f1c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:30.981673 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a1291-7ccf-441c-b57d-71913d9f1c30 None admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:30.982049 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a1291-7ccf-441c-b57d-71913d9f1c30 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:30.982382 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a1291-7ccf-441c-b57d-71913d9f1c30 None admin] PATH_INFO: `/v3/projects/aa11ab8229b642ce962193b74001d3bc/users/48c3e33f24ff4d66a72f06848d769ac7/roles/715d866890d14923b11470d544b4f960` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:30.998821 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2a1291-7ccf-441c-b57d-71913d9f1c30 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa11ab8229b642ce962193b74001d3bc, user_id=48c3e33f24ff4d66a72f06848d769ac7, role_id=715d866890d14923b11470d544b4f960)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:31.000257 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2a1291-7ccf-441c-b57d-71913d9f1c30 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:31.072377 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 306/608] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 21 22:35:30 2022] PUT /identity/v3/projects/aa11ab8229b642ce962193b74001d3bc/users/150f42651f754fda83f25fe3b408bce9/roles/715d866890d14923b11470d544b4f960 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:35:31.073261 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4d320c-3f89-4851-a6dd-37f001b07974 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:31.106698 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 303/609] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 21 22:35:30 2022] PUT /identity/v3/projects/aa11ab8229b642ce962193b74001d3bc/users/48c3e33f24ff4d66a72f06848d769ac7/roles/715d866890d14923b11470d544b4f960 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:31.108902 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4754085e-61ee-45e4-82fa-dde12d4aefe5 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:31.109278 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4754085e-61ee-45e4-82fa-dde12d4aefe5 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:31.109631 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4754085e-61ee-45e4-82fa-dde12d4aefe5 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:31.116997 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 304/610] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:31.119530 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2e0f4b-ea28-4495-b4f6-420815ac98c7 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:31.119900 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2e0f4b-ea28-4495-b4f6-420815ac98c7 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:31.120252 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2e0f4b-ea28-4495-b4f6-420815ac98c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:31.121642 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:31.144607 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-9d2e0f4b-ea28-4495-b4f6-420815ac98c7 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:31.146905 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-9d2e0f4b-ea28-4495-b4f6-420815ac98c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:31.244700 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 305/611] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:35:31.257681 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4d320c-3f89-4851-a6dd-37f001b07974 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:31.284629 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2e0f4b-ea28-4495-b4f6-420815ac98c7 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:31.311573 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11115896-1203-43c1-a77b-34f765401a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:31.312677 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11115896-1203-43c1-a77b-34f765401a0c None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:31.313128 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11115896-1203-43c1-a77b-34f765401a0c None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:31.313535 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11115896-1203-43c1-a77b-34f765401a0c None admin] PATH_INFO: `/v3/roles/service` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:31.325602 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11115896-1203-43c1-a77b-34f765401a0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:35:31.333736 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-11115896-1203-43c1-a77b-34f765401a0c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:35:31.333736 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:31.333736 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:31.333736 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:31.333736 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:31.333736 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:31.333736 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:31.333736 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:31.333736 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:31.333736 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:31.333736 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:31.333736 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:31.333736 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:35:31.333736 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:35:31.333736 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:35:31.333736 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:35:31.335462 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:35:31.335462 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:35:31.335462 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:35:31.335462 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:35:31.335462 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:31.335462 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:31.335462 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:31.335462 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:31.335462 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:31.335462 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:31.335462 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:31.335462 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:31.335462 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:31.335462 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:31.335462 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:31.335462 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:31.336484 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:31.336484 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:31.336484 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:31.336484 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:31.336484 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:35:31.336484 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:35:31.336484 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:35:31.336484 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:35:31.336484 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:35:31.336484 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:35:31.336484 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:35:31.336484 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:31.338839 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 307/612] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 21 22:35:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:31.344520 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11115896-1203-43c1-a77b-34f765401a0c None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:31.394260 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:31.410248 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2e0f4b-ea28-4495-b4f6-420815ac98c7 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:31.448555 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd12cc4-8212-4c08-9658-8086b058c56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:31.449425 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd12cc4-8212-4c08-9658-8086b058c56b None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:31.449818 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd12cc4-8212-4c08-9658-8086b058c56b None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:31.450105 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd12cc4-8212-4c08-9658-8086b058c56b None admin] PATH_INFO: `/v3/roles/service` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:31.452939 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:31.460052 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11115896-1203-43c1-a77b-34f765401a0c None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:31.462247 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd12cc4-8212-4c08-9658-8086b058c56b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:35:31.470021 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-8dd12cc4-8212-4c08-9658-8086b058c56b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:35:31.470021 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:31.470021 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:31.470021 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:31.470021 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:31.470021 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:31.470021 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:31.470021 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:31.470021 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:31.470021 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:31.470021 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:31.470021 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:31.470021 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:35:31.470021 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:35:31.470021 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:35:31.470021 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:35:31.471008 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:35:31.471008 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:35:31.471008 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:35:31.471008 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:35:31.471008 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:31.471008 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:31.471008 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:31.471008 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:31.471008 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:31.471008 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:31.471008 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:31.471008 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:31.471008 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:31.471008 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:31.471008 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:31.471008 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:31.471833 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:31.471833 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:31.471833 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:31.471833 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:31.471833 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:35:31.471833 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:35:31.471833 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:35:31.471833 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:35:31.471833 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:35:31.471833 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:35:31.471833 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:35:31.471833 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:31.471833 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 306/613] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 21 22:35:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:35:31.480535 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd12cc4-8212-4c08-9658-8086b058c56b None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:31.510367 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fac96b-d8e6-4688-8377-81d58d4413dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:31.512098 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fac96b-d8e6-4688-8377-81d58d4413dc None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:31.513364 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fac96b-d8e6-4688-8377-81d58d4413dc None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:31.513915 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fac96b-d8e6-4688-8377-81d58d4413dc None admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:31.515620 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fac96b-d8e6-4688-8377-81d58d4413dc None admin] RBAC: Adding query filter params (name=service) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:31.516185 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fac96b-d8e6-4688-8377-81d58d4413dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:31.520091 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fac96b-d8e6-4688-8377-81d58d4413dc None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:31.535449 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 308/614] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 21 22:35:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:31.540869 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fac96b-d8e6-4688-8377-81d58d4413dc None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:31.694201 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd12cc4-8212-4c08-9658-8086b058c56b None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:31.744436 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fac96b-d8e6-4688-8377-81d58d4413dc None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:31.763267 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdb1dc8-0a9b-44dc-8650-2751df5cbd85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:31.763869 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb1dc8-0a9b-44dc-8650-2751df5cbd85 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:31.764206 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb1dc8-0a9b-44dc-8650-2751df5cbd85 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:31.764487 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb1dc8-0a9b-44dc-8650-2751df5cbd85 None admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:31.765272 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdb1dc8-0a9b-44dc-8650-2751df5cbd85 None admin] RBAC: Adding query filter params (name=service) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:31.765556 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdb1dc8-0a9b-44dc-8650-2751df5cbd85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:31.766951 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdb1dc8-0a9b-44dc-8650-2751df5cbd85 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:31.775987 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 307/615] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 21 22:35:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:31.782399 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdb1dc8-0a9b-44dc-8650-2751df5cbd85 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:31.798396 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:31.823557 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f966d528-e9c5-4cf9-9b84-418e285a0ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:31.824774 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f966d528-e9c5-4cf9-9b84-418e285a0ab1 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:31.825296 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f966d528-e9c5-4cf9-9b84-418e285a0ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:31.825775 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f966d528-e9c5-4cf9-9b84-418e285a0ab1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:31.849230 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f966d528-e9c5-4cf9-9b84-418e285a0ab1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:31.856334 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f966d528-e9c5-4cf9-9b84-418e285a0ab1 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:31.891026 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-f966d528-e9c5-4cf9-9b84-418e285a0ab1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:31.891026 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:31.891026 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:31.891026 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:31.891026 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:31.891026 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:31.891026 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:31.891026 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:31.891026 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:31.891026 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:31.891026 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:31.891026 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:31.891026 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:31.891026 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:31.891026 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:31.891984 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:31.891984 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:31.891984 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:31.891984 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:31.891984 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:31.891984 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:31.891984 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:31.891984 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:31.891984 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:31.891984 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:31.891984 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:31.891984 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:31.891984 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:31.891984 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:31.891984 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:31.891984 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:31.891984 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:31.892860 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:31.892860 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:31.892860 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:31.892860 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:31.892860 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 309/616] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:35:31.897875 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f966d528-e9c5-4cf9-9b84-418e285a0ab1 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:31.910347 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:31.926216 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdb1dc8-0a9b-44dc-8650-2751df5cbd85 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:31.993161 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1673253-6ac6-4bcc-891c-889700cc6449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:31.993968 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1673253-6ac6-4bcc-891c-889700cc6449 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:31.994246 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1673253-6ac6-4bcc-891c-889700cc6449 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:31.994516 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1673253-6ac6-4bcc-891c-889700cc6449 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:32.012901 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1673253-6ac6-4bcc-891c-889700cc6449 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:32.014712 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1673253-6ac6-4bcc-891c-889700cc6449 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:32.034689 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-f1673253-6ac6-4bcc-891c-889700cc6449 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:32.034689 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:32.034689 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:32.034689 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:32.034689 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:32.034689 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:32.034689 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:32.034689 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:32.034689 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:32.034689 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:32.034689 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:32.034689 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:32.034689 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:32.034689 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:32.034689 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:32.035637 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:32.035637 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:32.035637 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:32.035637 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:32.035637 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:32.035637 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:32.035637 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:32.035637 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:32.035637 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:32.035637 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:32.035637 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:32.035637 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:32.035637 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:32.035637 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:32.035637 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:32.035637 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:32.035637 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:32.036515 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:32.036515 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:32.036515 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:32.036515 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:32.039837 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 308/617] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:32.046159 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1673253-6ac6-4bcc-891c-889700cc6449 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:32.068693 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f966d528-e9c5-4cf9-9b84-418e285a0ab1 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:32.132887 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfe6531-6874-4f45-8ef9-ecf12532e109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:32.134203 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfe6531-6874-4f45-8ef9-ecf12532e109 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:32.139156 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfe6531-6874-4f45-8ef9-ecf12532e109 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:32.139446 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfe6531-6874-4f45-8ef9-ecf12532e109 None admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:32.140199 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfe6531-6874-4f45-8ef9-ecf12532e109 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:32.140512 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfe6531-6874-4f45-8ef9-ecf12532e109 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:32.141912 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfe6531-6874-4f45-8ef9-ecf12532e109 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:32.143128 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:32.166704 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 310/618] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:35:32.172040 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfe6531-6874-4f45-8ef9-ecf12532e109 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:32.182361 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1673253-6ac6-4bcc-891c-889700cc6449 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:32.220202 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7de0f0-b8b3-4101-95f9-3512c44e594a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:32.221280 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7de0f0-b8b3-4101-95f9-3512c44e594a None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:32.221821 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7de0f0-b8b3-4101-95f9-3512c44e594a None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:32.222301 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7de0f0-b8b3-4101-95f9-3512c44e594a None admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:32.223438 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7de0f0-b8b3-4101-95f9-3512c44e594a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:32.223953 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7de0f0-b8b3-4101-95f9-3512c44e594a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:32.226157 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7de0f0-b8b3-4101-95f9-3512c44e594a None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:32.240579 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 309/619] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:32.246299 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7de0f0-b8b3-4101-95f9-3512c44e594a None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:32.319096 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:32.358946 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfe6531-6874-4f45-8ef9-ecf12532e109 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:32.369528 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7de0f0-b8b3-4101-95f9-3512c44e594a None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:32.395196 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:32.404632 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc665c0d-959a-40ff-a9e9-c96e2c701b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:32.405544 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc665c0d-959a-40ff-a9e9-c96e2c701b79 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:32.405837 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc665c0d-959a-40ff-a9e9-c96e2c701b79 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:32.406116 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc665c0d-959a-40ff-a9e9-c96e2c701b79 None admin] PATH_INFO: `/v3/users/glance` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:32.410726 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c738b18-25a4-4813-af76-0e57493576fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:32.411570 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c738b18-25a4-4813-af76-0e57493576fa None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:32.411856 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c738b18-25a4-4813-af76-0e57493576fa None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:32.412132 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c738b18-25a4-4813-af76-0e57493576fa None admin] PATH_INFO: `/v3/users/swift` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:32.415710 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc665c0d-959a-40ff-a9e9-c96e2c701b79 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:32.417401 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc665c0d-959a-40ff-a9e9-c96e2c701b79 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:32.420988 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c738b18-25a4-4813-af76-0e57493576fa None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:32.422493 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c738b18-25a4-4813-af76-0e57493576fa None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:32.434087 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-fc665c0d-959a-40ff-a9e9-c96e2c701b79 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 21 22:35:32.434087 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:32.434087 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:32.434087 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:32.434087 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:32.434087 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:32.434087 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:32.434087 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:32.434087 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:32.434087 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:32.434087 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:32.434087 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:32.434087 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:35:32.434087 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:35:32.434087 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:35:32.434087 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:35:32.435073 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:32.435073 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:32.435073 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:35:32.435073 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:35:32.435073 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:35:32.435073 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:35:32.435073 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:32.435073 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:32.435073 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:32.435073 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:32.435073 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:32.435073 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:32.435073 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:32.435073 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:32.435073 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:32.435073 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:32.435073 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:32.435073 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:32.436038 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:32.436038 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:32.436038 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:35:32.436038 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:35:32.436038 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:35:32.436038 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:35:32.436038 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:35:32.436038 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:35:32.436038 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 21 22:35:32.436038 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:32.438427 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-6c738b18-25a4-4813-af76-0e57493576fa None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 22:35:32.438427 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:32.438427 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:32.438427 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:32.438427 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:32.438427 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:32.438427 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:32.438427 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:32.438427 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:32.438427 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:32.438427 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:32.438427 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:32.438427 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:35:32.438427 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:35:32.438427 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:35:32.438427 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:35:32.439547 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:32.439547 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:32.439547 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:35:32.439547 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:35:32.439547 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:35:32.439547 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:35:32.439547 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:32.439547 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:32.439547 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:32.439547 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:32.439547 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:32.439547 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:32.439547 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:32.439547 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:32.439547 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:32.439547 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:32.439547 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:32.439547 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:32.440703 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:32.440703 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:32.440703 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:35:32.440703 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:35:32.440703 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:35:32.440703 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:35:32.440703 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:35:32.440703 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:35:32.440703 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 22:35:32.440703 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:32.443165 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 311/621] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:35:32 2022] GET /identity/v3/users/glance => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:35:32.443280 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 310/621] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 22:35:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:32.445298 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4758a5-ebbc-4205-acb6-1e9105a8c7d8 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:32.445589 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4758a5-ebbc-4205-acb6-1e9105a8c7d8 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:32.445869 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4758a5-ebbc-4205-acb6-1e9105a8c7d8 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:32.450816 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 311/622] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:32.454441 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc665c0d-959a-40ff-a9e9-c96e2c701b79 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:32.455452 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4758a5-ebbc-4205-acb6-1e9105a8c7d8 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:32.577819 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc665c0d-959a-40ff-a9e9-c96e2c701b79 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:32.582615 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4758a5-ebbc-4205-acb6-1e9105a8c7d8 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:32.608000 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:32.616800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14aed88-1a09-4833-a0e9-e685f1e1bddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:32.617465 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14aed88-1a09-4833-a0e9-e685f1e1bddf None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:32.617748 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14aed88-1a09-4833-a0e9-e685f1e1bddf None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:32.618023 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14aed88-1a09-4833-a0e9-e685f1e1bddf None admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:32.619013 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14aed88-1a09-4833-a0e9-e685f1e1bddf None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:32.619323 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14aed88-1a09-4833-a0e9-e685f1e1bddf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:32.621004 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14aed88-1a09-4833-a0e9-e685f1e1bddf None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:32.622196 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422721d7-4d45-4182-b900-291cf089f5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:32.623142 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422721d7-4d45-4182-b900-291cf089f5d4 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:32.623576 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422721d7-4d45-4182-b900-291cf089f5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:32.627807 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422721d7-4d45-4182-b900-291cf089f5d4 None admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:32.627807 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422721d7-4d45-4182-b900-291cf089f5d4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:32.627807 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422721d7-4d45-4182-b900-291cf089f5d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:32.627807 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422721d7-4d45-4182-b900-291cf089f5d4 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:32.645256 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 312/623] 10.222.0.52 () {56 vars in 1137 bytes} [Thu Jul 21 22:35:32 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:32.646802 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76e369-9e0e-4b92-b285-3443def07fbe None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:32.647120 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76e369-9e0e-4b92-b285-3443def07fbe None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:32.648397 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76e369-9e0e-4b92-b285-3443def07fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:32.649336 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 312/624] 10.222.0.52 () {56 vars in 1135 bytes} [Thu Jul 21 22:35:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:32.650984 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422721d7-4d45-4182-b900-291cf089f5d4 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:32.666301 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-7c76e369-9e0e-4b92-b285-3443def07fbe None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:32.667664 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-7c76e369-9e0e-4b92-b285-3443def07fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:32.696539 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:32.740456 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 313/625] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:35:32.741566 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c76e369-9e0e-4b92-b285-3443def07fbe None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:32.759256 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422721d7-4d45-4182-b900-291cf089f5d4 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:32.796310 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd13af61-add5-4b38-b45e-e762d5d9eef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:32.797107 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13af61-add5-4b38-b45e-e762d5d9eef7 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:32.797494 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13af61-add5-4b38-b45e-e762d5d9eef7 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:32.798204 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13af61-add5-4b38-b45e-e762d5d9eef7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:32.848889 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c76e369-9e0e-4b92-b285-3443def07fbe None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:32.869239 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:32.872836 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd13af61-add5-4b38-b45e-e762d5d9eef7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:32.874416 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd13af61-add5-4b38-b45e-e762d5d9eef7 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:32.889057 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd303d92-ba04-4501-ae5d-e8253d23eeaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:32.890146 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd303d92-ba04-4501-ae5d-e8253d23eeaa None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:32.890598 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd303d92-ba04-4501-ae5d-e8253d23eeaa None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:32.891086 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd303d92-ba04-4501-ae5d-e8253d23eeaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:32.902880 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd303d92-ba04-4501-ae5d-e8253d23eeaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:32.904550 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd303d92-ba04-4501-ae5d-e8253d23eeaa None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:32.918281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:32.925540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-bd303d92-ba04-4501-ae5d-e8253d23eeaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:32.925540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:32.925540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:32.925540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:32.925540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:32.925540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:32.925540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:32.925540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:32.925540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:32.925540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:32.925540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:32.925540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:32.925540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:32.925540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:32.925540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:32.926497 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:32.926497 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:32.926497 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:32.926497 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:32.926497 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:32.926497 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:32.926497 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:32.926497 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:32.926497 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:32.926497 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:32.926497 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:32.926497 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:32.926497 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:32.926497 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:32.926497 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:32.926497 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:32.926497 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:32.927430 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:32.927430 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:32.927430 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:32.927430 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:32.929799 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 314/626] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:32.930755 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd303d92-ba04-4501-ae5d-e8253d23eeaa None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:32.985159 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 313/627] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:35:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:35:32.986341 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd13af61-add5-4b38-b45e-e762d5d9eef7 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:33.040476 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd303d92-ba04-4501-ae5d-e8253d23eeaa None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:33.077747 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923bace4-0a42-4706-8c90-0238c0dc927d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:33.078965 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923bace4-0a42-4706-8c90-0238c0dc927d None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:33.079439 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923bace4-0a42-4706-8c90-0238c0dc927d None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:33.079887 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923bace4-0a42-4706-8c90-0238c0dc927d None admin] PATH_INFO: `/v3/roles/service` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:33.091483 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923bace4-0a42-4706-8c90-0238c0dc927d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:35:33.094794 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd13af61-add5-4b38-b45e-e762d5d9eef7 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:33.100113 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-923bace4-0a42-4706-8c90-0238c0dc927d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:35:33.100113 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:33.100113 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:33.100113 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:33.100113 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:33.100113 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:33.100113 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:33.100113 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:33.100113 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:33.100113 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:33.100113 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:33.100113 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:33.100113 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:35:33.100113 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:35:33.100113 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:35:33.100113 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:35:33.101033 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:35:33.101033 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:35:33.101033 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:35:33.101033 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:35:33.101033 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:33.101033 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:33.101033 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:33.101033 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:33.101033 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:33.101033 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:33.101033 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:33.101033 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:33.101033 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:33.101033 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:33.101033 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:33.101033 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:33.101937 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:33.101937 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:33.101937 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:33.101937 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:33.101937 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:35:33.101937 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:35:33.101937 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:35:33.101937 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:35:33.101937 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:35:33.101937 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:35:33.101937 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:35:33.101937 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:33.104929 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 315/628] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 21 22:35:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:33.106016 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923bace4-0a42-4706-8c90-0238c0dc927d None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:33.134004 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae9e0dc-ef6d-478f-94dd-355a2ade33d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:33.135153 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9e0dc-ef6d-478f-94dd-355a2ade33d8 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:33.135653 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9e0dc-ef6d-478f-94dd-355a2ade33d8 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:33.136003 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9e0dc-ef6d-478f-94dd-355a2ade33d8 None admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:33.136809 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae9e0dc-ef6d-478f-94dd-355a2ade33d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:33.137190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae9e0dc-ef6d-478f-94dd-355a2ade33d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:33.138800 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae9e0dc-ef6d-478f-94dd-355a2ade33d8 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:33.145506 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:33.149400 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:33.153048 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 314/629] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:33.153765 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae9e0dc-ef6d-478f-94dd-355a2ade33d8 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:33.223331 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923bace4-0a42-4706-8c90-0238c0dc927d None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:33.262803 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1fc427-1149-40e6-ba14-a47305b6a46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:33.263155 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae9e0dc-ef6d-478f-94dd-355a2ade33d8 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:33.264142 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1fc427-1149-40e6-ba14-a47305b6a46d None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:33.264730 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1fc427-1149-40e6-ba14-a47305b6a46d None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:33.265222 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1fc427-1149-40e6-ba14-a47305b6a46d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:33.276967 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1fc427-1149-40e6-ba14-a47305b6a46d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:33.279265 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1fc427-1149-40e6-ba14-a47305b6a46d None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:33.299216 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-ad1fc427-1149-40e6-ba14-a47305b6a46d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:33.299216 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:33.299216 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:33.299216 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:33.299216 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:33.299216 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:33.299216 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:33.299216 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:33.299216 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:33.299216 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:33.299216 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:33.299216 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:33.299216 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:33.299216 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:33.299216 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:33.300150 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:33.300150 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:33.300150 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:33.300150 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:33.300150 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:33.300150 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:33.300150 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:33.300150 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:33.300150 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:33.300150 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:33.300150 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:33.300150 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:33.300150 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:33.300150 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:33.300150 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:33.300150 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:33.300150 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:33.301017 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:33.301017 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:33.301017 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:33.301017 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:33.301251 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da51364b-7ab3-4513-9d28-0fec50994ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:33.301251 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da51364b-7ab3-4513-9d28-0fec50994ee6 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:33.301251 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da51364b-7ab3-4513-9d28-0fec50994ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:33.301251 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da51364b-7ab3-4513-9d28-0fec50994ee6 None admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:33.301971 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da51364b-7ab3-4513-9d28-0fec50994ee6 None admin] RBAC: Adding query filter params (name=service) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:33.302064 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da51364b-7ab3-4513-9d28-0fec50994ee6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:33.303471 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da51364b-7ab3-4513-9d28-0fec50994ee6 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:33.304030 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 316/630] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:33.305103 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1fc427-1149-40e6-ba14-a47305b6a46d None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:33.313066 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 315/631] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 21 22:35:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:33.314007 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da51364b-7ab3-4513-9d28-0fec50994ee6 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:33.392099 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:33.406385 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:33.426122 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da51364b-7ab3-4513-9d28-0fec50994ee6 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:33.427396 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1fc427-1149-40e6-ba14-a47305b6a46d None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:33.461581 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27325ae6-6dc8-41b9-a45e-2ae0e60800d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:33.462367 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27325ae6-6dc8-41b9-a45e-2ae0e60800d7 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:33.462646 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27325ae6-6dc8-41b9-a45e-2ae0e60800d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:33.462947 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27325ae6-6dc8-41b9-a45e-2ae0e60800d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:33.464926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a0bd3c-99b7-4999-af15-2b0f0000b2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:33.465901 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a0bd3c-99b7-4999-af15-2b0f0000b2cc None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:33.466192 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a0bd3c-99b7-4999-af15-2b0f0000b2cc None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:33.466472 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a0bd3c-99b7-4999-af15-2b0f0000b2cc None admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:33.467340 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a0bd3c-99b7-4999-af15-2b0f0000b2cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:33.467869 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a0bd3c-99b7-4999-af15-2b0f0000b2cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:33.469544 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a0bd3c-99b7-4999-af15-2b0f0000b2cc None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:33.470020 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27325ae6-6dc8-41b9-a45e-2ae0e60800d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:33.471458 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27325ae6-6dc8-41b9-a45e-2ae0e60800d7 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:33.485528 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-27325ae6-6dc8-41b9-a45e-2ae0e60800d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:35:33.485528 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:33.485528 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:33.485528 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:33.485528 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:33.485528 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:33.485528 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:33.485528 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:33.485528 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:33.485528 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:33.485528 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:33.485528 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:33.485528 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:35:33.485528 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:35:33.485528 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:35:33.486552 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:35:33.486552 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:33.486552 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:33.486552 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:33.486552 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:33.486552 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:33.486552 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:33.486552 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:33.486552 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:33.486552 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:33.486552 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:33.486552 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:33.486552 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:33.486552 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:33.486552 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:33.486552 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:33.486552 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:33.487527 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:35:33.487527 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:35:33.487527 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:35:33.487527 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:35:33.487527 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:35:33.487527 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:35:33.487527 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:35:33.487527 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:33.489774 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 316/632] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:33.491140 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a0bd3c-99b7-4999-af15-2b0f0000b2cc None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:33.491718 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 317/633] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:35:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:33.493260 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e80332-1988-4f7e-841d-a15ee88dd017 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:33.493994 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e80332-1988-4f7e-841d-a15ee88dd017 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:33.494487 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e80332-1988-4f7e-841d-a15ee88dd017 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:33.497231 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 318/634] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:33.499531 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e80332-1988-4f7e-841d-a15ee88dd017 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:33.600074 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a0bd3c-99b7-4999-af15-2b0f0000b2cc None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:33.603664 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e80332-1988-4f7e-841d-a15ee88dd017 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:33.632962 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab81d96-bc3a-4be0-95c3-7b51b112daa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:33.633746 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab81d96-bc3a-4be0-95c3-7b51b112daa0 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:33.634025 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab81d96-bc3a-4be0-95c3-7b51b112daa0 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:33.634298 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab81d96-bc3a-4be0-95c3-7b51b112daa0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:33.636819 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01426bd-ec9b-477e-84f1-cb6f651ac496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:33.637604 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01426bd-ec9b-477e-84f1-cb6f651ac496 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:33.637890 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01426bd-ec9b-477e-84f1-cb6f651ac496 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:33.638170 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01426bd-ec9b-477e-84f1-cb6f651ac496 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:33.644131 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab81d96-bc3a-4be0-95c3-7b51b112daa0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:33.644772 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01426bd-ec9b-477e-84f1-cb6f651ac496 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:33.645513 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab81d96-bc3a-4be0-95c3-7b51b112daa0 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:33.646186 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01426bd-ec9b-477e-84f1-cb6f651ac496 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:33.660719 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-b01426bd-ec9b-477e-84f1-cb6f651ac496 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:35:33.660719 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:33.660719 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:33.660719 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:33.660719 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:33.660719 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:33.660719 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:33.660719 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:33.660719 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:33.660719 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:33.660719 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:33.660719 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:33.660719 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:35:33.660719 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:35:33.660719 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:35:33.661800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:35:33.661800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:33.661800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:33.661800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:33.661800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:33.661800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:33.661800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:33.661800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:33.661800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:33.661800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:33.661800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:33.661800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:33.661800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:33.661800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:33.661800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:33.661800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:33.661800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:33.662726 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:35:33.662726 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:35:33.662726 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:35:33.662726 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:35:33.662726 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:35:33.662726 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:35:33.662726 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:35:33.662726 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:33.662726 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 319/635] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:35:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:33.662726 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01426bd-ec9b-477e-84f1-cb6f651ac496 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:33.663314 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-4ab81d96-bc3a-4be0-95c3-7b51b112daa0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:33.663314 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:33.663314 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:33.663314 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:33.663314 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:33.663314 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:33.663314 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:33.663314 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:33.663314 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:33.663314 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:33.663314 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:33.663314 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:33.663314 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:33.663314 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:33.663314 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:33.664112 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:33.664112 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:33.664112 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:33.664112 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:33.664112 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:33.664112 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:33.664112 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:33.664112 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:33.664112 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:33.664112 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:33.664112 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:33.664112 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:33.664112 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:33.664112 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:33.664112 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:33.664112 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:33.664112 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:33.665080 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:33.665080 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:33.665080 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:33.665080 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:33.665080 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 317/636] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:33.665611 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc1349a-a48d-4020-8946-fd22a11a2d85 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:33.665893 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc1349a-a48d-4020-8946-fd22a11a2d85 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:33.666170 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc1349a-a48d-4020-8946-fd22a11a2d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:33.682898 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-7cc1349a-a48d-4020-8946-fd22a11a2d85 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:33.684088 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-7cc1349a-a48d-4020-8946-fd22a11a2d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:33.742426 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:33.750505 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 318/637] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:35:33.751823 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ca2b7-8bc1-480c-b9bf-525ad04ddce5 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:33.752110 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ca2b7-8bc1-480c-b9bf-525ad04ddce5 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:33.752392 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ca2b7-8bc1-480c-b9bf-525ad04ddce5 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:33.753771 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 319/638] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:33.754320 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4ca2b7-8bc1-480c-b9bf-525ad04ddce5 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:33.775546 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01426bd-ec9b-477e-84f1-cb6f651ac496 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:33.778499 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:33.810956 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd5c240-b196-4868-b331-49fe4bcbd384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:33.811610 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd5c240-b196-4868-b331-49fe4bcbd384 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:33.811897 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd5c240-b196-4868-b331-49fe4bcbd384 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:33.812169 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd5c240-b196-4868-b331-49fe4bcbd384 None admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:33.812954 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd5c240-b196-4868-b331-49fe4bcbd384 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:33.813282 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd5c240-b196-4868-b331-49fe4bcbd384 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:33.814679 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd5c240-b196-4868-b331-49fe4bcbd384 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:33.827933 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 320/639] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 22:35:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:33.828722 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd5c240-b196-4868-b331-49fe4bcbd384 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:33.858594 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4ca2b7-8bc1-480c-b9bf-525ad04ddce5 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:33.861678 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:33.887461 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:33.895297 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e83d3-13c6-4cf7-8741-fea9e7ac9a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:33.895897 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e83d3-13c6-4cf7-8741-fea9e7ac9a84 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:33.896183 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e83d3-13c6-4cf7-8741-fea9e7ac9a84 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:33.896457 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e83d3-13c6-4cf7-8741-fea9e7ac9a84 None admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:33.897276 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397e83d3-13c6-4cf7-8741-fea9e7ac9a84 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:33.897585 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397e83d3-13c6-4cf7-8741-fea9e7ac9a84 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:33.899107 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397e83d3-13c6-4cf7-8741-fea9e7ac9a84 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:33.910720 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 320/640] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 22:35:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:33.911962 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fca921-f921-43aa-8e9a-89c87fb82055 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:33.912248 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fca921-f921-43aa-8e9a-89c87fb82055 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:33.912525 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fca921-f921-43aa-8e9a-89c87fb82055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:33.929041 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-05fca921-f921-43aa-8e9a-89c87fb82055 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:33.929231 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd5c240-b196-4868-b331-49fe4bcbd384 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:33.930220 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-05fca921-f921-43aa-8e9a-89c87fb82055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:33.962050 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c63a5-bc3a-4d68-92ea-10fa151a7d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:33.962708 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c63a5-bc3a-4d68-92ea-10fa151a7d79 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:33.963090 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c63a5-bc3a-4d68-92ea-10fa151a7d79 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:33.963368 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c63a5-bc3a-4d68-92ea-10fa151a7d79 None admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:33.964083 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56c63a5-bc3a-4d68-92ea-10fa151a7d79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:33.964398 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56c63a5-bc3a-4d68-92ea-10fa151a7d79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:33.965748 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56c63a5-bc3a-4d68-92ea-10fa151a7d79 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:33.977623 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 321/641] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:33.978386 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c63a5-bc3a-4d68-92ea-10fa151a7d79 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:33.999139 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 321/642] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:35:34.000024 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fca921-f921-43aa-8e9a-89c87fb82055 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:34.085354 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c63a5-bc3a-4d68-92ea-10fa151a7d79 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:34.091204 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:34.097907 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fca921-f921-43aa-8e9a-89c87fb82055 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:34.105715 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:34.119833 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9376abe8-9276-42a5-bc40-cf29a87bc704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:34.123853 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9376abe8-9276-42a5-bc40-cf29a87bc704 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:34.123853 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9376abe8-9276-42a5-bc40-cf29a87bc704 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:34.123853 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9376abe8-9276-42a5-bc40-cf29a87bc704 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:34.131358 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9376abe8-9276-42a5-bc40-cf29a87bc704 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:35:34.131934 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65774957-bb29-44b7-a8c2-2cf29355da3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:34.132819 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65774957-bb29-44b7-a8c2-2cf29355da3e None admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:34.133102 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65774957-bb29-44b7-a8c2-2cf29355da3e None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:34.133381 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65774957-bb29-44b7-a8c2-2cf29355da3e None admin] PATH_INFO: `/v3/projects/aa11ab8229b642ce962193b74001d3bc/users/ba7eef4e1b27476ab4016449d9c3a4b9/roles/715d866890d14923b11470d544b4f960` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:34.139056 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-9376abe8-9276-42a5-bc40-cf29a87bc704 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:35:34.139056 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:34.139056 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:34.139056 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:34.139056 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:34.139056 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:34.139056 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:34.139056 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:34.139056 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:34.139056 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:34.139056 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:34.139056 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:34.139056 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:35:34.139056 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:35:34.139056 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:35:34.139056 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:35:34.140308 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:35:34.140308 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:35:34.140308 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:35:34.140308 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:35:34.140308 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:34.140308 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:34.140308 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:34.140308 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:34.140308 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:34.140308 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:34.140308 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:34.140308 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:34.140308 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:34.140308 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:34.140308 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:34.140308 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:34.141389 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:34.141389 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:34.141389 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:34.141389 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:34.141389 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:35:34.141389 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:35:34.141389 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:35:34.141389 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:35:34.141389 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:35:34.141389 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:35:34.141389 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:35:34.141389 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:34.143090 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 322/643] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 21 22:35:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:34.144073 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9376abe8-9276-42a5-bc40-cf29a87bc704 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:34.152105 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65774957-bb29-44b7-a8c2-2cf29355da3e None admin] RBAC: Authorizing `identity:create_grant(project_id=aa11ab8229b642ce962193b74001d3bc, user_id=ba7eef4e1b27476ab4016449d9c3a4b9, role_id=715d866890d14923b11470d544b4f960)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:34.152105 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65774957-bb29-44b7-a8c2-2cf29355da3e None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:34.223778 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 322/644] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 21 22:35:33 2022] PUT /identity/v3/projects/aa11ab8229b642ce962193b74001d3bc/users/ba7eef4e1b27476ab4016449d9c3a4b9/roles/715d866890d14923b11470d544b4f960 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:34.224576 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65774957-bb29-44b7-a8c2-2cf29355da3e None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:34.248111 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9376abe8-9276-42a5-bc40-cf29a87bc704 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:34.285959 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24990cd3-565c-4100-a145-4861fb4c469b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:34.286724 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24990cd3-565c-4100-a145-4861fb4c469b None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:34.287098 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24990cd3-565c-4100-a145-4861fb4c469b None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:34.287443 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24990cd3-565c-4100-a145-4861fb4c469b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:34.288793 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24990cd3-565c-4100-a145-4861fb4c469b None admin] RBAC: Adding query filter params (role.id=715d866890d14923b11470d544b4f960, scope.project.id=aa11ab8229b642ce962193b74001d3bc, user.id=927778d005fd48a4b4b8fd3c4419bb36) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:34.289109 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24990cd3-565c-4100-a145-4861fb4c469b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:34.290583 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24990cd3-565c-4100-a145-4861fb4c469b None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:34.297671 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 323/645] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 22:35:34 2022] GET /identity/v3/role_assignments?user.id=927778d005fd48a4b4b8fd3c4419bb36&scope.project.id=aa11ab8229b642ce962193b74001d3bc&role.id=715d866890d14923b11470d544b4f960 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:34.298619 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24990cd3-565c-4100-a145-4861fb4c469b None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:34.365556 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:34.370982 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65774957-bb29-44b7-a8c2-2cf29355da3e None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:34.420842 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9c0dd3-25f9-4b38-bb0b-8c29099dddaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:34.421907 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c0dd3-25f9-4b38-bb0b-8c29099dddaf None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:34.422361 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c0dd3-25f9-4b38-bb0b-8c29099dddaf None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:34.422935 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c0dd3-25f9-4b38-bb0b-8c29099dddaf None admin] PATH_INFO: `/v3/users/nova` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:34.433639 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9c0dd3-25f9-4b38-bb0b-8c29099dddaf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:34.435678 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9c0dd3-25f9-4b38-bb0b-8c29099dddaf None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:34.445476 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24990cd3-565c-4100-a145-4861fb4c469b None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:34.454043 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-db9c0dd3-25f9-4b38-bb0b-8c29099dddaf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:35:34.454043 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:34.454043 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:34.454043 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:34.454043 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:34.454043 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:34.454043 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:34.454043 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:34.454043 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:34.454043 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:34.454043 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:34.454043 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:34.454043 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:35:34.454043 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:35:34.454043 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:35:34.454043 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:35:34.455123 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:34.455123 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:34.455123 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:35:34.455123 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:35:34.455123 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:35:34.455123 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:35:34.455123 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:34.455123 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:34.455123 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:34.455123 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:34.455123 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:34.455123 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:34.455123 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:34.455123 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:34.455123 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:34.455123 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:34.455123 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:34.455123 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:34.456240 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:34.456240 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:34.456240 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:35:34.456240 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:35:34.456240 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:35:34.456240 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:35:34.456240 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:35:34.456240 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:35:34.456240 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:35:34.456240 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:34.456240 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 323/646] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Jul 21 22:35:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:34.456240 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9c0dd3-25f9-4b38-bb0b-8c29099dddaf None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:34.461788 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:34.501684 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf73240a-9ec9-47cd-9bb6-1df28057321c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:34.502591 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf73240a-9ec9-47cd-9bb6-1df28057321c None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:34.502937 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf73240a-9ec9-47cd-9bb6-1df28057321c None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:34.503419 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf73240a-9ec9-47cd-9bb6-1df28057321c None admin] PATH_INFO: `/v3/roles/service` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:34.521224 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf73240a-9ec9-47cd-9bb6-1df28057321c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:35:34.530024 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-bf73240a-9ec9-47cd-9bb6-1df28057321c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:35:34.530024 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:34.530024 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:34.530024 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:34.530024 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:34.530024 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:34.530024 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:34.530024 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:34.530024 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:34.530024 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:34.530024 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:34.530024 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:34.530024 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:35:34.530024 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:35:34.530024 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:35:34.530024 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:35:34.531068 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:35:34.531068 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:35:34.531068 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:35:34.531068 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:35:34.531068 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:34.531068 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:34.531068 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:34.531068 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:34.531068 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:34.531068 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:34.531068 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:34.531068 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:34.531068 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:34.531068 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:34.531068 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:34.531068 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:34.531988 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:34.531988 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:34.531988 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:34.531988 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:34.531988 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:35:34.531988 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:35:34.531988 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:35:34.531988 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:35:34.531988 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:35:34.531988 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:35:34.531988 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:35:34.531988 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:34.532624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 324/647] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 21 22:35:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:34.533113 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf73240a-9ec9-47cd-9bb6-1df28057321c None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:34.612570 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9c0dd3-25f9-4b38-bb0b-8c29099dddaf None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:34.651669 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b61eb8-6b81-4f4e-9cfc-2c72e066a871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:34.652338 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b61eb8-6b81-4f4e-9cfc-2c72e066a871 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:34.652621 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b61eb8-6b81-4f4e-9cfc-2c72e066a871 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:34.652894 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b61eb8-6b81-4f4e-9cfc-2c72e066a871 None admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:34.653639 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b61eb8-6b81-4f4e-9cfc-2c72e066a871 None admin] RBAC: Adding query filter params (name=service) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:34.653945 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b61eb8-6b81-4f4e-9cfc-2c72e066a871 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:34.655348 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b61eb8-6b81-4f4e-9cfc-2c72e066a871 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:34.660940 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf73240a-9ec9-47cd-9bb6-1df28057321c None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:34.663436 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:34.669539 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 324/648] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 21 22:35:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:34.670689 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b61eb8-6b81-4f4e-9cfc-2c72e066a871 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:34.697862 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20b9318-e9b4-4dde-bd73-c0e86dba0f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:34.699059 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b9318-e9b4-4dde-bd73-c0e86dba0f07 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:34.699664 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b9318-e9b4-4dde-bd73-c0e86dba0f07 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:34.700400 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b9318-e9b4-4dde-bd73-c0e86dba0f07 None admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:34.701450 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20b9318-e9b4-4dde-bd73-c0e86dba0f07 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:34.701878 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20b9318-e9b4-4dde-bd73-c0e86dba0f07 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:34.703450 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20b9318-e9b4-4dde-bd73-c0e86dba0f07 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:34.729340 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 325/649] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Jul 21 22:35:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:34.730426 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20b9318-e9b4-4dde-bd73-c0e86dba0f07 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:34.775177 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:34.806018 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b61eb8-6b81-4f4e-9cfc-2c72e066a871 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:34.849862 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29788727-19b3-4e38-b08b-3f58e8bb932b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:34.851117 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29788727-19b3-4e38-b08b-3f58e8bb932b None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:34.851406 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29788727-19b3-4e38-b08b-3f58e8bb932b None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:34.851686 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29788727-19b3-4e38-b08b-3f58e8bb932b None admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:34.852207 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20b9318-e9b4-4dde-bd73-c0e86dba0f07 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:34.852703 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29788727-19b3-4e38-b08b-3f58e8bb932b None admin] RBAC: Adding query filter params (name=service) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:34.853014 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29788727-19b3-4e38-b08b-3f58e8bb932b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:34.854575 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29788727-19b3-4e38-b08b-3f58e8bb932b None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:34.863801 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 325/650] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 21 22:35:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:34.864672 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29788727-19b3-4e38-b08b-3f58e8bb932b None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:34.887905 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f391f-4811-443b-b4a9-cb28aa381aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:34.888697 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f391f-4811-443b-b4a9-cb28aa381aaf None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:34.888978 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f391f-4811-443b-b4a9-cb28aa381aaf None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:34.889250 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f391f-4811-443b-b4a9-cb28aa381aaf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:34.899774 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5f391f-4811-443b-b4a9-cb28aa381aaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:34.901216 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5f391f-4811-443b-b4a9-cb28aa381aaf None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:34.919684 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-9a5f391f-4811-443b-b4a9-cb28aa381aaf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:34.919684 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:34.919684 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:34.919684 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:34.919684 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:34.919684 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:34.919684 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:34.919684 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:34.919684 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:34.919684 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:34.919684 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:34.919684 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:34.919684 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:34.919684 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:34.919684 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:34.920620 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:34.920620 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:34.920620 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:34.920620 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:34.920620 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:34.920620 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:34.920620 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:34.920620 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:34.920620 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:34.920620 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:34.920620 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:34.920620 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:34.920620 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:34.920620 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:34.920620 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:34.920620 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:34.920620 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:34.921511 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:34.921511 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:34.921511 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:34.921511 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:34.921511 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 326/651] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:34.922224 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f391f-4811-443b-b4a9-cb28aa381aaf None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:34.941839 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:34.983534 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29788727-19b3-4e38-b08b-3f58e8bb932b None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:35.013618 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:35.023689 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5465aa87-a134-4751-b549-029b692ef218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:35.024551 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465aa87-a134-4751-b549-029b692ef218 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:35.025006 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465aa87-a134-4751-b549-029b692ef218 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:35.025496 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465aa87-a134-4751-b549-029b692ef218 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:35.031891 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f391f-4811-443b-b4a9-cb28aa381aaf None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:35.067689 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ed183-b51d-41f7-b05f-6aa6a9163a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:35.068483 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ed183-b51d-41f7-b05f-6aa6a9163a86 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:35.068766 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ed183-b51d-41f7-b05f-6aa6a9163a86 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:35.069559 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ed183-b51d-41f7-b05f-6aa6a9163a86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:35.079917 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66ed183-b51d-41f7-b05f-6aa6a9163a86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:35.081469 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66ed183-b51d-41f7-b05f-6aa6a9163a86 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:35.100356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-d66ed183-b51d-41f7-b05f-6aa6a9163a86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:35.100356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:35.100356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:35.100356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:35.100356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:35.100356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:35.100356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:35.100356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:35.100356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:35.100356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:35.100356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:35.100356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:35.100356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:35.100356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:35.100356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:35.101447 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:35.101447 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:35.101447 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:35.101447 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:35.101447 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:35.101447 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:35.101447 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:35.101447 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:35.101447 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:35.101447 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:35.101447 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:35.101447 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:35.101447 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:35.101447 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:35.101447 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:35.101447 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:35.101447 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:35.103033 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:35.103033 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:35.103033 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:35.103033 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:35.103033 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 327/652] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:35.103033 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ed183-b51d-41f7-b05f-6aa6a9163a86 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:35.106928 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5465aa87-a134-4751-b549-029b692ef218 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:35.108599 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5465aa87-a134-4751-b549-029b692ef218 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:35.170685 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:35.207761 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ed183-b51d-41f7-b05f-6aa6a9163a86 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:35.221975 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 326/653] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:35:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:35:35.223223 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5465aa87-a134-4751-b549-029b692ef218 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:35.244215 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b021ad89-5930-43c1-8d32-e0ec2ec9b90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:35.244896 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021ad89-5930-43c1-8d32-e0ec2ec9b90c None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:35.245244 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021ad89-5930-43c1-8d32-e0ec2ec9b90c None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:35.245613 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021ad89-5930-43c1-8d32-e0ec2ec9b90c None admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:35.246697 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b021ad89-5930-43c1-8d32-e0ec2ec9b90c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:35.247115 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b021ad89-5930-43c1-8d32-e0ec2ec9b90c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:35.248553 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b021ad89-5930-43c1-8d32-e0ec2ec9b90c None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:35.258045 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:35.264763 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 328/654] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:35.273754 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b021ad89-5930-43c1-8d32-e0ec2ec9b90c None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:35.343566 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5465aa87-a134-4751-b549-029b692ef218 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:35.384872 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd46687f-dd18-4ac3-ae3a-2dc7ccdbe3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:35.385545 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46687f-dd18-4ac3-ae3a-2dc7ccdbe3ac None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:35.385828 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46687f-dd18-4ac3-ae3a-2dc7ccdbe3ac None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:35.386102 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46687f-dd18-4ac3-ae3a-2dc7ccdbe3ac None admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:35.386842 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd46687f-dd18-4ac3-ae3a-2dc7ccdbe3ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:35.387002 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b021ad89-5930-43c1-8d32-e0ec2ec9b90c None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:35.387148 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd46687f-dd18-4ac3-ae3a-2dc7ccdbe3ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:35.388618 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd46687f-dd18-4ac3-ae3a-2dc7ccdbe3ac None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:35.402035 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 327/655] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:35.403029 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd46687f-dd18-4ac3-ae3a-2dc7ccdbe3ac None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:35.416756 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:35.425441 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438733b7-6cd1-41ef-b4f4-e26bd1e0b714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:35.426445 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438733b7-6cd1-41ef-b4f4-e26bd1e0b714 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:35.426585 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438733b7-6cd1-41ef-b4f4-e26bd1e0b714 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:35.426859 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438733b7-6cd1-41ef-b4f4-e26bd1e0b714 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:35.437671 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438733b7-6cd1-41ef-b4f4-e26bd1e0b714 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:35.439162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438733b7-6cd1-41ef-b4f4-e26bd1e0b714 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:35.458743 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-438733b7-6cd1-41ef-b4f4-e26bd1e0b714 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:35.458743 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:35.458743 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:35.458743 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:35.458743 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:35.458743 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:35.458743 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:35.458743 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:35.458743 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:35.458743 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:35.458743 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:35.458743 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:35.458743 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:35.458743 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:35.458743 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:35.460553 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:35.460553 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:35.460553 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:35.460553 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:35.460553 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:35.460553 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:35.460553 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:35.460553 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:35.460553 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:35.460553 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:35.460553 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:35.460553 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:35.460553 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:35.460553 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:35.460553 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:35.460553 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:35.460553 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:35.461494 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:35.461494 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:35.461494 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:35.461494 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:35.465147 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 329/656] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:35.466086 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438733b7-6cd1-41ef-b4f4-e26bd1e0b714 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:35.504323 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:35.517407 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd46687f-dd18-4ac3-ae3a-2dc7ccdbe3ac None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:35.557748 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c2c45-73e7-45e3-9eb3-189fc33667c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:35.558971 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c2c45-73e7-45e3-9eb3-189fc33667c2 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:35.559399 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c2c45-73e7-45e3-9eb3-189fc33667c2 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:35.559814 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c2c45-73e7-45e3-9eb3-189fc33667c2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:35.568419 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80c2c45-73e7-45e3-9eb3-189fc33667c2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:35.570728 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80c2c45-73e7-45e3-9eb3-189fc33667c2 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:35.586303 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438733b7-6cd1-41ef-b4f4-e26bd1e0b714 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:35.586797 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-c80c2c45-73e7-45e3-9eb3-189fc33667c2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 22:35:35.586797 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:35.586797 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:35.586797 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:35.586797 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:35.586797 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:35.586797 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:35.586797 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:35.586797 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:35.586797 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:35.586797 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:35.586797 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:35.586797 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:35:35.586797 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:35:35.586797 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:35:35.586797 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:35:35.587795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:35.587795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:35.587795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:35:35.587795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:35:35.587795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:35:35.587795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:35:35.587795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:35.587795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:35.587795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:35.587795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:35.587795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:35.587795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:35.587795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:35.587795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:35.587795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:35.587795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:35.587795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:35.587795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:35.588716 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:35.588716 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:35.588716 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:35:35.588716 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:35:35.588716 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:35:35.588716 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:35:35.588716 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:35:35.588716 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:35:35.588716 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 22:35:35.588716 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:35.591409 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 328/657] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 21 22:35:35 2022] GET /identity/v3/users/neutron => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:35.592380 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c2c45-73e7-45e3-9eb3-189fc33667c2 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:35.624049 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c5441d-3d21-4fbe-87c8-30061ed677ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:35.624958 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5441d-3d21-4fbe-87c8-30061ed677ed None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:35.625338 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5441d-3d21-4fbe-87c8-30061ed677ed None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:35.625659 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5441d-3d21-4fbe-87c8-30061ed677ed None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:35.635440 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c5441d-3d21-4fbe-87c8-30061ed677ed None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:35.637047 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c5441d-3d21-4fbe-87c8-30061ed677ed None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:35.653452 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-f3c5441d-3d21-4fbe-87c8-30061ed677ed None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 22:35:35.653452 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:35.653452 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:35.653452 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:35.653452 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:35.653452 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:35.653452 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:35.653452 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:35.653452 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:35.653452 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:35.653452 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:35.653452 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:35.653452 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:35:35.653452 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:35:35.653452 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:35:35.653452 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:35:35.654407 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:35.654407 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:35.654407 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:35:35.654407 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:35:35.654407 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:35:35.654407 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:35:35.654407 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:35.654407 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:35.654407 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:35.654407 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:35.654407 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:35.654407 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:35.654407 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:35.654407 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:35.654407 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:35.654407 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:35.654407 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:35.654407 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:35.655370 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:35.655370 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:35.655370 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:35:35.655370 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:35:35.655370 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:35:35.655370 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:35:35.655370 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:35:35.655370 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:35:35.655370 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 22:35:35.655370 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:35.655370 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 330/658] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:35:35 2022] GET /identity/v3/users/cinder => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:35.655965 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c5441d-3d21-4fbe-87c8-30061ed677ed None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:35.699029 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:35.706450 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c2c45-73e7-45e3-9eb3-189fc33667c2 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:35.742563 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd2fe0b-2522-4160-b879-ccda7d0bff90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:35.743481 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd2fe0b-2522-4160-b879-ccda7d0bff90 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:35.744017 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd2fe0b-2522-4160-b879-ccda7d0bff90 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:35.744455 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd2fe0b-2522-4160-b879-ccda7d0bff90 None admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:35.745428 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd2fe0b-2522-4160-b879-ccda7d0bff90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:35.745876 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd2fe0b-2522-4160-b879-ccda7d0bff90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:35.747404 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd2fe0b-2522-4160-b879-ccda7d0bff90 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:35.761924 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 329/659] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:35.762963 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd2fe0b-2522-4160-b879-ccda7d0bff90 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:35.769795 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c5441d-3d21-4fbe-87c8-30061ed677ed None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:35.774931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:35.808660 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4116b7-4c85-4076-960a-d818d05fddaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:35.809695 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4116b7-4c85-4076-960a-d818d05fddaf None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:35.810259 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4116b7-4c85-4076-960a-d818d05fddaf None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:35.810745 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4116b7-4c85-4076-960a-d818d05fddaf None admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:35.812488 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4116b7-4c85-4076-960a-d818d05fddaf None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:35.813578 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4116b7-4c85-4076-960a-d818d05fddaf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:35.815283 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4116b7-4c85-4076-960a-d818d05fddaf None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:35.840540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 331/660] 10.222.0.52 () {56 vars in 1139 bytes} [Thu Jul 21 22:35:35 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:35.841719 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4116b7-4c85-4076-960a-d818d05fddaf None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:35.871798 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd2fe0b-2522-4160-b879-ccda7d0bff90 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:35.908781 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d8757-dd60-41f9-b6d1-bef10cc3932e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:35.909463 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d8757-dd60-41f9-b6d1-bef10cc3932e None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:35.909752 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d8757-dd60-41f9-b6d1-bef10cc3932e None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:35.910150 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d8757-dd60-41f9-b6d1-bef10cc3932e None admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:35.911294 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d8757-dd60-41f9-b6d1-bef10cc3932e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:35.911404 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d8757-dd60-41f9-b6d1-bef10cc3932e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:35.912906 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d8757-dd60-41f9-b6d1-bef10cc3932e None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:35.932354 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 330/661] 10.222.0.52 () {56 vars in 1137 bytes} [Thu Jul 21 22:35:35 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:35.933366 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d8757-dd60-41f9-b6d1-bef10cc3932e None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:35.947807 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:35.953976 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4116b7-4c85-4076-960a-d818d05fddaf None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:35.991541 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b5661-f499-4ff1-8a54-5fdbcd89edea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:35.992490 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b5661-f499-4ff1-8a54-5fdbcd89edea None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:35.992926 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b5661-f499-4ff1-8a54-5fdbcd89edea None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:35.993499 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b5661-f499-4ff1-8a54-5fdbcd89edea None admin] PATH_INFO: `/v3/projects/service` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:36.001517 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0b5661-f499-4ff1-8a54-5fdbcd89edea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:36.003334 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0b5661-f499-4ff1-8a54-5fdbcd89edea None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:36.005881 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:36.023911 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-6f0b5661-f499-4ff1-8a54-5fdbcd89edea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:35:36.023911 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:36.023911 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:36.023911 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:36.023911 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:36.023911 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:36.023911 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:36.023911 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:36.023911 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:36.023911 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:36.023911 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:36.023911 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:36.023911 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:35:36.023911 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:35:36.023911 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:35:36.024884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:35:36.024884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:36.024884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:36.024884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:36.024884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:36.024884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:36.024884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:36.024884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:36.024884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:36.024884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:36.024884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:36.024884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:36.024884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:36.024884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:36.024884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:36.024884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:36.024884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:36.025772 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:35:36.025772 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:35:36.025772 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:35:36.025772 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:35:36.025772 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:35:36.025772 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:35:36.025772 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:35:36.025772 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:36.028366 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 332/662] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:35:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:36.029456 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b5661-f499-4ff1-8a54-5fdbcd89edea None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:36.059035 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d8757-dd60-41f9-b6d1-bef10cc3932e None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:36.102143 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052b371b-ad27-46aa-9594-59ef8703ef4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:36.103208 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052b371b-ad27-46aa-9594-59ef8703ef4d None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:36.103663 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052b371b-ad27-46aa-9594-59ef8703ef4d None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:36.104084 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052b371b-ad27-46aa-9594-59ef8703ef4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:36.161812 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b5661-f499-4ff1-8a54-5fdbcd89edea None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:36.179605 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052b371b-ad27-46aa-9594-59ef8703ef4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:36.181442 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052b371b-ad27-46aa-9594-59ef8703ef4d None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:36.198673 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00660b86-1aa9-4660-bf14-415dfed3bc1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:36.199241 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00660b86-1aa9-4660-bf14-415dfed3bc1e None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:36.199661 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00660b86-1aa9-4660-bf14-415dfed3bc1e None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:36.200068 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00660b86-1aa9-4660-bf14-415dfed3bc1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:36.226037 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:36.258176 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:36.275312 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00660b86-1aa9-4660-bf14-415dfed3bc1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:36.276924 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00660b86-1aa9-4660-bf14-415dfed3bc1e None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:36.300045 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 331/663] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:35:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:35:36.301214 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052b371b-ad27-46aa-9594-59ef8703ef4d None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:36.381777 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 333/664] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:35:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:35:36.382798 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00660b86-1aa9-4660-bf14-415dfed3bc1e None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:36.412068 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052b371b-ad27-46aa-9594-59ef8703ef4d None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:36.439964 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:36.448292 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ad5df-88e1-4ba0-8cbc-4b6ff6259959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:36.449027 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ad5df-88e1-4ba0-8cbc-4b6ff6259959 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:36.452985 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ad5df-88e1-4ba0-8cbc-4b6ff6259959 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:36.452985 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ad5df-88e1-4ba0-8cbc-4b6ff6259959 None admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:36.452985 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1ad5df-88e1-4ba0-8cbc-4b6ff6259959 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:36.452985 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1ad5df-88e1-4ba0-8cbc-4b6ff6259959 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:36.452985 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1ad5df-88e1-4ba0-8cbc-4b6ff6259959 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:36.469640 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 332/665] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 22:35:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:36.469640 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd806664-06da-460b-8693-7ce44cfae449 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:36.469640 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd806664-06da-460b-8693-7ce44cfae449 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:36.469640 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd806664-06da-460b-8693-7ce44cfae449 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:36.469640 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 333/666] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:36.469640 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd806664-06da-460b-8693-7ce44cfae449 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:36.497122 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00660b86-1aa9-4660-bf14-415dfed3bc1e None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:36.508555 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:36.535012 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99db7d98-0fdb-4d88-b29b-0e69ccfbfd39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:36.535721 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db7d98-0fdb-4d88-b29b-0e69ccfbfd39 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:36.536004 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db7d98-0fdb-4d88-b29b-0e69ccfbfd39 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:36.536280 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db7d98-0fdb-4d88-b29b-0e69ccfbfd39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:36.548401 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99db7d98-0fdb-4d88-b29b-0e69ccfbfd39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:36.549955 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99db7d98-0fdb-4d88-b29b-0e69ccfbfd39 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:36.569280 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-99db7d98-0fdb-4d88-b29b-0e69ccfbfd39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:36.569280 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:36.569280 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:36.569280 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:36.569280 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:36.569280 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:36.569280 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:36.569280 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:36.569280 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:36.569280 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:36.569280 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:36.569280 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:36.569280 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:36.569280 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:36.569280 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:36.570220 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:36.570220 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:36.570220 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:36.570220 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:36.570220 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:36.570220 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:36.570220 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:36.570220 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:36.570220 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:36.570220 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:36.570220 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:36.570220 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:36.570220 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:36.570220 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:36.570220 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:36.570220 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:36.570220 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:36.571558 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:36.571558 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:36.571558 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:36.571558 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:36.571558 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 334/667] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:36.571558 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99db7d98-0fdb-4d88-b29b-0e69ccfbfd39 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:36.595664 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd806664-06da-460b-8693-7ce44cfae449 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:36.631169 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679fc916-ff58-4623-b063-586cefd38b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:36.631995 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679fc916-ff58-4623-b063-586cefd38b04 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:36.632285 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679fc916-ff58-4623-b063-586cefd38b04 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:36.632564 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679fc916-ff58-4623-b063-586cefd38b04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:36.643170 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679fc916-ff58-4623-b063-586cefd38b04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:36.644606 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679fc916-ff58-4623-b063-586cefd38b04 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:36.661825 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-679fc916-ff58-4623-b063-586cefd38b04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:36.661825 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:36.661825 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:36.661825 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:36.661825 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:36.661825 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:36.661825 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:36.661825 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:36.661825 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:36.661825 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:36.661825 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:36.661825 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:36.661825 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:36.661825 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:36.661825 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:36.663422 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:36.663422 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:36.663422 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:36.663422 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:36.663422 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:36.663422 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:36.663422 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:36.663422 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:36.663422 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:36.663422 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:36.663422 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:36.663422 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:36.663422 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:36.663422 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:36.663422 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:36.663422 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:36.663422 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:36.664315 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:36.664315 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:36.664315 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:36.664315 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:36.664315 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 334/668] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:36.665071 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b8913-15eb-4830-b064-0b7b0e7fea3a None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:36.665318 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b8913-15eb-4830-b064-0b7b0e7fea3a None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:36.665594 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b8913-15eb-4830-b064-0b7b0e7fea3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:36.677517 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99db7d98-0fdb-4d88-b29b-0e69ccfbfd39 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:36.681957 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-198b8913-15eb-4830-b064-0b7b0e7fea3a None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:36.683172 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-198b8913-15eb-4830-b064-0b7b0e7fea3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:36.687837 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:36.712736 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196bfb12-718b-430c-9fef-5ceb2addd9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:36.713388 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196bfb12-718b-430c-9fef-5ceb2addd9a6 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:36.713678 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196bfb12-718b-430c-9fef-5ceb2addd9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:36.713958 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196bfb12-718b-430c-9fef-5ceb2addd9a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:36.714820 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196bfb12-718b-430c-9fef-5ceb2addd9a6 None admin] RBAC: Adding query filter params (role.id=715d866890d14923b11470d544b4f960, scope.project.id=aa11ab8229b642ce962193b74001d3bc, user.id=6bfdb655a2a44b33ab724daaf73a294b) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:36.715137 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196bfb12-718b-430c-9fef-5ceb2addd9a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:36.716565 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196bfb12-718b-430c-9fef-5ceb2addd9a6 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:36.722299 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 335/669] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 22:35:36 2022] GET /identity/v3/role_assignments?user.id=6bfdb655a2a44b33ab724daaf73a294b&scope.project.id=aa11ab8229b642ce962193b74001d3bc&role.id=715d866890d14923b11470d544b4f960 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:36.723101 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196bfb12-718b-430c-9fef-5ceb2addd9a6 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:36.758926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 335/670] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:35:36.758926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678756bd-30ff-42c9-92e2-38790fb33d34 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:36.758926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678756bd-30ff-42c9-92e2-38790fb33d34 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:36.758926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678756bd-30ff-42c9-92e2-38790fb33d34 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:36.758926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 336/671] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:36.758926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678756bd-30ff-42c9-92e2-38790fb33d34 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:36.783988 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:36.850362 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196bfb12-718b-430c-9fef-5ceb2addd9a6 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:36.890934 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe0dff3-ffc7-4c85-a9cb-5ef9c38f63ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:36.891599 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe0dff3-ffc7-4c85-a9cb-5ef9c38f63ff None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:36.891878 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe0dff3-ffc7-4c85-a9cb-5ef9c38f63ff None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:36.892149 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe0dff3-ffc7-4c85-a9cb-5ef9c38f63ff None admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:36.892869 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe0dff3-ffc7-4c85-a9cb-5ef9c38f63ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:36.893086 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678756bd-30ff-42c9-92e2-38790fb33d34 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:36.893499 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe0dff3-ffc7-4c85-a9cb-5ef9c38f63ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:36.894517 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe0dff3-ffc7-4c85-a9cb-5ef9c38f63ff None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:36.912404 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 336/672] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:36.912404 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d1ddf-31ef-4289-b5fe-c6627964b335 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:36.912404 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d1ddf-31ef-4289-b5fe-c6627964b335 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:36.912404 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d1ddf-31ef-4289-b5fe-c6627964b335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:36.929101 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-ba6d1ddf-31ef-4289-b5fe-c6627964b335 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:36.930397 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-ba6d1ddf-31ef-4289-b5fe-c6627964b335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:36.945260 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d3e965-52b3-4183-8fc8-4cf275fe9209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:36.946150 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3e965-52b3-4183-8fc8-4cf275fe9209 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:36.946446 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3e965-52b3-4183-8fc8-4cf275fe9209 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:36.946727 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3e965-52b3-4183-8fc8-4cf275fe9209 None admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:36.947570 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d3e965-52b3-4183-8fc8-4cf275fe9209 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:36.947879 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d3e965-52b3-4183-8fc8-4cf275fe9209 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:36.949270 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d3e965-52b3-4183-8fc8-4cf275fe9209 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:36.961143 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 337/673] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:36.961788 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d3e965-52b3-4183-8fc8-4cf275fe9209 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:36.962164 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:37.009333 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 337/674] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:35:37.010628 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6d1ddf-31ef-4289-b5fe-c6627964b335 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:37.043522 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:37.079367 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d3e965-52b3-4183-8fc8-4cf275fe9209 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:37.116517 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb7d6fd-65a9-44e4-8775-53be5a72c341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:37.117232 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb7d6fd-65a9-44e4-8775-53be5a72c341 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:37.117510 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb7d6fd-65a9-44e4-8775-53be5a72c341 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:37.117780 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb7d6fd-65a9-44e4-8775-53be5a72c341 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:37.121879 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6d1ddf-31ef-4289-b5fe-c6627964b335 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:37.128555 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb7d6fd-65a9-44e4-8775-53be5a72c341 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:35:37.136734 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-3fb7d6fd-65a9-44e4-8775-53be5a72c341 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:35:37.136734 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:37.136734 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:37.136734 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:37.136734 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:37.136734 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:37.136734 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:37.136734 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:37.136734 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:37.136734 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:37.136734 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:37.136734 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:37.136734 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:35:37.136734 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:35:37.136734 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:35:37.136734 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:35:37.137657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:35:37.137657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:35:37.137657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:35:37.137657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:35:37.137657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:37.137657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:37.137657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:37.137657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:37.137657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:37.137657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:37.137657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:37.137657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:37.137657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:37.137657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:37.137657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:37.137657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:37.138506 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:37.138506 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:37.138506 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:37.138506 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:37.138506 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:35:37.138506 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:35:37.138506 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:35:37.138506 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:35:37.138506 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:35:37.138506 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:35:37.138506 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:35:37.138506 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:37.138506 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 338/675] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 21 22:35:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:37.139254 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb7d6fd-65a9-44e4-8775-53be5a72c341 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:37.167627 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1bcdf3-6a23-479d-b065-233950dd90dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:37.168421 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1bcdf3-6a23-479d-b065-233950dd90dc None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:37.168718 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1bcdf3-6a23-479d-b065-233950dd90dc None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:37.168968 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1bcdf3-6a23-479d-b065-233950dd90dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:37.175723 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1bcdf3-6a23-479d-b065-233950dd90dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:37.177150 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1bcdf3-6a23-479d-b065-233950dd90dc None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:37.191316 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-0a1bcdf3-6a23-479d-b065-233950dd90dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:35:37.191316 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:37.191316 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:37.191316 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:37.191316 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:37.191316 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:37.191316 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:37.191316 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:37.191316 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:37.191316 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:37.191316 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:37.191316 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:37.191316 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:35:37.191316 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:35:37.191316 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:35:37.192231 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:35:37.192231 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:37.192231 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:37.192231 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:37.192231 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:37.192231 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:37.192231 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:37.192231 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:37.192231 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:37.192231 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:37.192231 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:37.192231 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:37.192231 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:37.192231 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:37.192231 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:37.192231 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:37.192231 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:37.193129 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:35:37.193129 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:35:37.193129 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:35:37.193129 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:35:37.193129 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:35:37.193129 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:35:37.193129 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:35:37.193129 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:37.193129 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 338/676] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:35:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:37.193625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1bcdf3-6a23-479d-b065-233950dd90dc None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:37.223274 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:37.261182 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb7d6fd-65a9-44e4-8775-53be5a72c341 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:37.289714 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:37.297742 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b3b9a6-e799-42d6-b766-dc1ec02336b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:37.298535 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3b9a6-e799-42d6-b766-dc1ec02336b5 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:37.305418 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3b9a6-e799-42d6-b766-dc1ec02336b5 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:37.305418 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3b9a6-e799-42d6-b766-dc1ec02336b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:37.306057 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b3b9a6-e799-42d6-b766-dc1ec02336b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:37.307545 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b3b9a6-e799-42d6-b766-dc1ec02336b5 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:37.311375 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1bcdf3-6a23-479d-b065-233950dd90dc None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:37.324405 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-84b3b9a6-e799-42d6-b766-dc1ec02336b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:35:37.324405 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:37.324405 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:37.324405 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:37.324405 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:37.324405 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:37.324405 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:37.324405 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:37.324405 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:37.324405 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:37.324405 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:37.324405 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:37.324405 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:35:37.324405 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:35:37.324405 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:35:37.325303 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:35:37.325303 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:37.325303 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:37.325303 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:37.325303 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:37.325303 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:37.325303 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:37.325303 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:37.325303 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:37.325303 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:37.325303 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:37.325303 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:37.325303 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:37.325303 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:37.325303 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:37.325303 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:37.325303 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:37.326195 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:35:37.326195 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:35:37.326195 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:35:37.326195 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:35:37.326195 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:35:37.326195 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:35:37.326195 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:35:37.326195 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:37.326195 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 339/677] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:35:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:37.326729 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b3b9a6-e799-42d6-b766-dc1ec02336b5 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:37.353669 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea152654-314d-41d8-87cd-e0e7b9e513ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:37.354468 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea152654-314d-41d8-87cd-e0e7b9e513ce None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:37.354757 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea152654-314d-41d8-87cd-e0e7b9e513ce None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:37.355086 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea152654-314d-41d8-87cd-e0e7b9e513ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:37.365962 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea152654-314d-41d8-87cd-e0e7b9e513ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:37.367478 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea152654-314d-41d8-87cd-e0e7b9e513ce None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:37.386087 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-ea152654-314d-41d8-87cd-e0e7b9e513ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:37.386087 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:37.386087 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:37.386087 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:37.386087 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:37.386087 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:37.386087 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:37.386087 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:37.386087 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:37.386087 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:37.386087 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:37.386087 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:37.386087 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:37.386087 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:37.386087 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:37.387012 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:37.387012 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:37.387012 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:37.387012 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:37.387012 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:37.387012 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:37.387012 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:37.387012 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:37.387012 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:37.387012 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:37.387012 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:37.387012 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:37.387012 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:37.387012 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:37.387012 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:37.387012 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:37.387012 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:37.387884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:37.387884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:37.387884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:37.387884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:37.387884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 339/678] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:37.388196 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea152654-314d-41d8-87cd-e0e7b9e513ce None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:37.440773 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b3b9a6-e799-42d6-b766-dc1ec02336b5 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:37.463468 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:37.482991 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6258e12a-436c-4f1f-bb0a-f93daa077dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:37.483672 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6258e12a-436c-4f1f-bb0a-f93daa077dfe None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:37.483959 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6258e12a-436c-4f1f-bb0a-f93daa077dfe None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:37.484241 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6258e12a-436c-4f1f-bb0a-f93daa077dfe None admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:37.484993 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6258e12a-436c-4f1f-bb0a-f93daa077dfe None admin] RBAC: Adding query filter params (name=service) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:37.485306 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6258e12a-436c-4f1f-bb0a-f93daa077dfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:37.486672 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6258e12a-436c-4f1f-bb0a-f93daa077dfe None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:37.490299 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea152654-314d-41d8-87cd-e0e7b9e513ce None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:37.505950 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 340/679] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 21 22:35:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:37.505950 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6258e12a-436c-4f1f-bb0a-f93daa077dfe None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:37.534746 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908540db-b0e2-41ad-807b-958d9d0b0b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:37.535456 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908540db-b0e2-41ad-807b-958d9d0b0b72 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:37.535742 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908540db-b0e2-41ad-807b-958d9d0b0b72 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:37.536016 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908540db-b0e2-41ad-807b-958d9d0b0b72 None admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:37.536811 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908540db-b0e2-41ad-807b-958d9d0b0b72 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:37.537126 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908540db-b0e2-41ad-807b-958d9d0b0b72 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:37.538593 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908540db-b0e2-41ad-807b-958d9d0b0b72 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:37.554337 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 340/680] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 22:35:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:37.555027 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908540db-b0e2-41ad-807b-958d9d0b0b72 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:37.561959 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:37.618842 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6258e12a-436c-4f1f-bb0a-f93daa077dfe None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:37.654649 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18555b9c-d717-4fa4-9754-d446c576a1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:37.655350 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18555b9c-d717-4fa4-9754-d446c576a1cb None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:37.655629 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18555b9c-d717-4fa4-9754-d446c576a1cb None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:37.655902 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18555b9c-d717-4fa4-9754-d446c576a1cb None admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:37.656679 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18555b9c-d717-4fa4-9754-d446c576a1cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:37.656988 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18555b9c-d717-4fa4-9754-d446c576a1cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:37.658511 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18555b9c-d717-4fa4-9754-d446c576a1cb None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:37.663006 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908540db-b0e2-41ad-807b-958d9d0b0b72 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:37.675619 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 341/681] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 22:35:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:37.675619 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18555b9c-d717-4fa4-9754-d446c576a1cb None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:37.701238 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717a9350-db6b-4766-9b3c-317d4a5b36fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:37.701898 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a9350-db6b-4766-9b3c-317d4a5b36fa None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:37.702183 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a9350-db6b-4766-9b3c-317d4a5b36fa None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:37.702484 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a9350-db6b-4766-9b3c-317d4a5b36fa None admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:37.703201 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717a9350-db6b-4766-9b3c-317d4a5b36fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:37.703510 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717a9350-db6b-4766-9b3c-317d4a5b36fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:37.704861 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717a9350-db6b-4766-9b3c-317d4a5b36fa None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:37.706414 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:37.717692 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 341/682] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:37.718501 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717a9350-db6b-4766-9b3c-317d4a5b36fa None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:37.783710 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18555b9c-d717-4fa4-9754-d446c576a1cb None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:37.796155 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:37.820654 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3356f-fab7-4f66-8e8c-2cbcf1ba5316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:37.821840 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3356f-fab7-4f66-8e8c-2cbcf1ba5316 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:37.822315 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3356f-fab7-4f66-8e8c-2cbcf1ba5316 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:37.822636 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3356f-fab7-4f66-8e8c-2cbcf1ba5316 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:37.825072 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717a9350-db6b-4766-9b3c-317d4a5b36fa None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:37.832934 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf3356f-fab7-4f66-8e8c-2cbcf1ba5316 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:37.834440 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf3356f-fab7-4f66-8e8c-2cbcf1ba5316 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:37.859073 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-cdf3356f-fab7-4f66-8e8c-2cbcf1ba5316 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:37.859073 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:37.859073 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:37.859073 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:37.859073 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:37.859073 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:37.859073 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:37.859073 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:37.859073 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:37.859073 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:37.859073 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:37.859073 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:37.859073 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:37.859073 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:37.859073 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:37.860056 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:37.860056 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:37.860056 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:37.860056 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:37.860056 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:37.860056 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:37.860056 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:37.860056 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:37.860056 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:37.860056 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:37.860056 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:37.860056 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:37.860056 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:37.860056 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:37.860056 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:37.860056 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:37.860056 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:37.860952 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:37.860952 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:37.860952 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:37.860952 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:37.860952 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 342/683] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:37.861235 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3356f-fab7-4f66-8e8c-2cbcf1ba5316 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:37.869597 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c73eee0-3832-49eb-ad6d-71cc575c4bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:37.870202 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c73eee0-3832-49eb-ad6d-71cc575c4bb5 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:37.870576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c73eee0-3832-49eb-ad6d-71cc575c4bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:37.871404 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c73eee0-3832-49eb-ad6d-71cc575c4bb5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:37.871867 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c73eee0-3832-49eb-ad6d-71cc575c4bb5 None admin] RBAC: Adding query filter params (role.id=715d866890d14923b11470d544b4f960, scope.project.id=aa11ab8229b642ce962193b74001d3bc, user.id=48c3e33f24ff4d66a72f06848d769ac7) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:37.872178 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c73eee0-3832-49eb-ad6d-71cc575c4bb5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:37.873764 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c73eee0-3832-49eb-ad6d-71cc575c4bb5 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:37.890942 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 342/684] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 22:35:37 2022] GET /identity/v3/role_assignments?user.id=48c3e33f24ff4d66a72f06848d769ac7&scope.project.id=aa11ab8229b642ce962193b74001d3bc&role.id=715d866890d14923b11470d544b4f960 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:37.890942 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c73eee0-3832-49eb-ad6d-71cc575c4bb5 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:37.961850 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:37.999546 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3356f-fab7-4f66-8e8c-2cbcf1ba5316 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:38.013169 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c73eee0-3832-49eb-ad6d-71cc575c4bb5 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:38.051901 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6890d396-a454-468c-bcad-9d45cae3c3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:38.052579 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890d396-a454-468c-bcad-9d45cae3c3d9 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:38.052864 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890d396-a454-468c-bcad-9d45cae3c3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:38.053174 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890d396-a454-468c-bcad-9d45cae3c3d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:38.054031 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6890d396-a454-468c-bcad-9d45cae3c3d9 None admin] RBAC: Adding query filter params (role.id=715d866890d14923b11470d544b4f960, scope.project.id=aa11ab8229b642ce962193b74001d3bc, user.id=150f42651f754fda83f25fe3b408bce9) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:38.054347 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6890d396-a454-468c-bcad-9d45cae3c3d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:38.055864 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6890d396-a454-468c-bcad-9d45cae3c3d9 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:38.059674 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2ab989-1078-46ff-be3b-2f1cc75044da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:38.060941 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ab989-1078-46ff-be3b-2f1cc75044da None admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:38.061428 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ab989-1078-46ff-be3b-2f1cc75044da None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:38.061919 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ab989-1078-46ff-be3b-2f1cc75044da None admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:38.062879 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2ab989-1078-46ff-be3b-2f1cc75044da None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:38.064545 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2ab989-1078-46ff-be3b-2f1cc75044da None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:38.070959 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 343/685] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 22:35:37 2022] GET /identity/v3/role_assignments?user.id=150f42651f754fda83f25fe3b408bce9&scope.project.id=aa11ab8229b642ce962193b74001d3bc&role.id=715d866890d14923b11470d544b4f960 => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:38.071975 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6890d396-a454-468c-bcad-9d45cae3c3d9 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:38.112833 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:38.162094 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 343/686] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 21 22:35:37 2022] POST /identity/v3/users => generated 254 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:38.223600 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6890d396-a454-468c-bcad-9d45cae3c3d9 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:38.269682 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9848917-bd7e-4b9c-8d75-1f0b7b0717a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:38.270377 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9848917-bd7e-4b9c-8d75-1f0b7b0717a7 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:38.270666 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9848917-bd7e-4b9c-8d75-1f0b7b0717a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:38.270980 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9848917-bd7e-4b9c-8d75-1f0b7b0717a7 None admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:38.271759 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9848917-bd7e-4b9c-8d75-1f0b7b0717a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:38.272101 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9848917-bd7e-4b9c-8d75-1f0b7b0717a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:38.273495 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9848917-bd7e-4b9c-8d75-1f0b7b0717a7 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:38.292062 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 344/687] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:38.297609 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2ab989-1078-46ff-be3b-2f1cc75044da None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:38.422268 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2ab989-1078-46ff-be3b-2f1cc75044da None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:38.437455 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:38.459138 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcf8eab-25e9-4547-9c75-2e8e9453f1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:38.460474 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcf8eab-25e9-4547-9c75-2e8e9453f1ff None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:38.460762 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcf8eab-25e9-4547-9c75-2e8e9453f1ff None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:38.461036 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcf8eab-25e9-4547-9c75-2e8e9453f1ff None admin] PATH_INFO: `/v3/users/swift` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:38.469716 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcf8eab-25e9-4547-9c75-2e8e9453f1ff None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:38.471217 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcf8eab-25e9-4547-9c75-2e8e9453f1ff None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:38.492126 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-2fcf8eab-25e9-4547-9c75-2e8e9453f1ff None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 22:35:38.492126 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:38.492126 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:38.492126 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:38.492126 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:38.492126 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:38.492126 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:38.492126 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:38.492126 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:38.492126 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:38.492126 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:38.492126 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:38.492126 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:35:38.492126 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:35:38.492126 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:35:38.492126 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:35:38.493189 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:38.493189 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:38.493189 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:35:38.493189 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:35:38.493189 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:35:38.493189 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:35:38.493189 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:38.493189 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:38.493189 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:38.493189 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:38.493189 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:38.493189 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:38.493189 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:38.493189 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:38.493189 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:38.493189 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:38.493189 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:38.493189 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:38.494114 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:38.494114 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:38.494114 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:35:38.494114 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:35:38.494114 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:35:38.494114 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:35:38.494114 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:35:38.494114 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:35:38.494114 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 22:35:38.494114 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:38.494114 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 344/688] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 22:35:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:38.506184 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9848917-bd7e-4b9c-8d75-1f0b7b0717a7 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:38.612743 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:38.629164 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9848917-bd7e-4b9c-8d75-1f0b7b0717a7 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:38.668725 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8831ef4d-4a5a-4847-b655-1db0b2d86e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:38.669641 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831ef4d-4a5a-4847-b655-1db0b2d86e94 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:38.670087 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831ef4d-4a5a-4847-b655-1db0b2d86e94 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:38.670512 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831ef4d-4a5a-4847-b655-1db0b2d86e94 None admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:38.671619 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8831ef4d-4a5a-4847-b655-1db0b2d86e94 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:38.672069 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8831ef4d-4a5a-4847-b655-1db0b2d86e94 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:38.673728 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8831ef4d-4a5a-4847-b655-1db0b2d86e94 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:38.692226 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 345/689] 10.222.0.52 () {56 vars in 1135 bytes} [Thu Jul 21 22:35:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:38.698593 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcf8eab-25e9-4547-9c75-2e8e9453f1ff None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:38.844122 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcf8eab-25e9-4547-9c75-2e8e9453f1ff None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:38.879820 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3526effb-a2ce-4c9d-b9c3-5e7fb0188f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:38.880478 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3526effb-a2ce-4c9d-b9c3-5e7fb0188f84 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:38.880765 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3526effb-a2ce-4c9d-b9c3-5e7fb0188f84 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:38.881121 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3526effb-a2ce-4c9d-b9c3-5e7fb0188f84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:38.887679 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f4561-eb66-47d8-96ba-d3899b78bd39 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:38.888172 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f4561-eb66-47d8-96ba-d3899b78bd39 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:38.888602 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f4561-eb66-47d8-96ba-d3899b78bd39 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:38.890352 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 346/690] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:38.895278 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6723c6-14c5-4f30-bdfd-ff8f12c2f582 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:38.895564 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6723c6-14c5-4f30-bdfd-ff8f12c2f582 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:38.895846 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6723c6-14c5-4f30-bdfd-ff8f12c2f582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:38.930273 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-3c6723c6-14c5-4f30-bdfd-ff8f12c2f582 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:38.933992 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-3c6723c6-14c5-4f30-bdfd-ff8f12c2f582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:38.953433 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:38.996051 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3526effb-a2ce-4c9d-b9c3-5e7fb0188f84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:38.998333 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3526effb-a2ce-4c9d-b9c3-5e7fb0188f84 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:39.066821 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 347/691] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:35:39.119975 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6723c6-14c5-4f30-bdfd-ff8f12c2f582 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:39.143077 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 345/692] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:35:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:35:39.151833 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3526effb-a2ce-4c9d-b9c3-5e7fb0188f84 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:39.225873 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:39.252688 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3526effb-a2ce-4c9d-b9c3-5e7fb0188f84 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:39.288854 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64759b25-a199-41b8-91e7-2ebeabdad55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:39.289648 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64759b25-a199-41b8-91e7-2ebeabdad55d None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:39.289932 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64759b25-a199-41b8-91e7-2ebeabdad55d None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:39.290203 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64759b25-a199-41b8-91e7-2ebeabdad55d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:39.301126 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64759b25-a199-41b8-91e7-2ebeabdad55d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:39.302567 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64759b25-a199-41b8-91e7-2ebeabdad55d None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:39.307127 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:39.321068 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-64759b25-a199-41b8-91e7-2ebeabdad55d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:39.321068 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:39.321068 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:39.321068 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:39.321068 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:39.321068 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:39.321068 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:39.321068 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:39.321068 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:39.321068 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:39.321068 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:39.321068 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:39.321068 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:39.321068 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:39.321068 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:39.326239 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:39.326239 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:39.326239 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:39.326239 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:39.326239 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:39.326239 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:39.326239 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:39.326239 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:39.326239 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:39.326239 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:39.326239 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:39.326239 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:39.326239 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:39.326239 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:39.326239 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:39.326239 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:39.326239 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:39.327193 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:39.327193 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:39.327193 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:39.327193 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:39.327193 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 346/693] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:39.328557 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64759b25-a199-41b8-91e7-2ebeabdad55d None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:39.365310 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6723c6-14c5-4f30-bdfd-ff8f12c2f582 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:39.427453 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d87e9e-bb5e-4215-b25d-de0f2cda580e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:39.428277 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d87e9e-bb5e-4215-b25d-de0f2cda580e None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:39.428559 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d87e9e-bb5e-4215-b25d-de0f2cda580e None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:39.428831 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d87e9e-bb5e-4215-b25d-de0f2cda580e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:39.451685 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d87e9e-bb5e-4215-b25d-de0f2cda580e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:35:39.461780 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-12d87e9e-bb5e-4215-b25d-de0f2cda580e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:35:39.461780 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:39.461780 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:39.461780 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:39.461780 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:39.461780 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:39.461780 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:39.461780 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:39.461780 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:39.461780 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:39.461780 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:39.461780 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:39.461780 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:35:39.461780 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:35:39.461780 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:35:39.461780 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:35:39.462795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:35:39.462795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:35:39.462795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:35:39.462795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:35:39.462795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:39.462795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:39.462795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:39.462795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:39.462795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:39.462795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:39.462795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:39.462795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:39.462795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:39.462795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:39.462795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:39.462795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:39.463647 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:39.463647 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:39.463647 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:39.463647 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:39.463647 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:35:39.463647 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:35:39.463647 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:35:39.463647 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:35:39.463647 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:35:39.463647 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:35:39.463647 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:35:39.463647 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:39.464438 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 348/694] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 22:35:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:39.470106 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d87e9e-bb5e-4215-b25d-de0f2cda580e None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:39.502492 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64759b25-a199-41b8-91e7-2ebeabdad55d None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:39.545456 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:39.550268 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b2f212-7254-46cd-bd29-b214e562f485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:39.551250 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2f212-7254-46cd-bd29-b214e562f485 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:39.551735 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2f212-7254-46cd-bd29-b214e562f485 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:39.552229 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2f212-7254-46cd-bd29-b214e562f485 None admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:39.553799 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b2f212-7254-46cd-bd29-b214e562f485 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:39.554326 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b2f212-7254-46cd-bd29-b214e562f485 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:39.556002 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b2f212-7254-46cd-bd29-b214e562f485 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:39.572735 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 347/695] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:35:39.579951 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b2f212-7254-46cd-bd29-b214e562f485 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:39.700945 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d87e9e-bb5e-4215-b25d-de0f2cda580e None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:39.738407 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b2f212-7254-46cd-bd29-b214e562f485 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:39.779492 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ea200-fa65-443c-a8b5-c5c0d96980ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:39.780092 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ea200-fa65-443c-a8b5-c5c0d96980ba None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:39.780381 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ea200-fa65-443c-a8b5-c5c0d96980ba None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:39.780661 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ea200-fa65-443c-a8b5-c5c0d96980ba None admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:39.781417 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9ea200-fa65-443c-a8b5-c5c0d96980ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:39.781726 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9ea200-fa65-443c-a8b5-c5c0d96980ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:39.783198 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9ea200-fa65-443c-a8b5-c5c0d96980ba None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:39.787070 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:39.800201 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 349/696] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:35:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:39.805528 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f614248d-4056-4247-a0fc-599a8a0edc53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:39.806323 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f614248d-4056-4247-a0fc-599a8a0edc53 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:39.806612 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f614248d-4056-4247-a0fc-599a8a0edc53 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:39.813309 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f614248d-4056-4247-a0fc-599a8a0edc53 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:39.813425 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ea200-fa65-443c-a8b5-c5c0d96980ba None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:39.828040 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f614248d-4056-4247-a0fc-599a8a0edc53 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:39.829492 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f614248d-4056-4247-a0fc-599a8a0edc53 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:39.851135 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-f614248d-4056-4247-a0fc-599a8a0edc53 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:35:39.851135 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:39.851135 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:39.851135 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:39.851135 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:39.851135 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:39.851135 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:39.851135 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:39.851135 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:39.851135 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:39.851135 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:39.851135 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:39.851135 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:35:39.851135 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:35:39.851135 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:35:39.852308 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:35:39.852308 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:39.852308 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:39.852308 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:39.852308 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:39.852308 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:39.852308 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:39.852308 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:39.852308 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:39.852308 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:39.852308 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:39.852308 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:39.852308 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:39.852308 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:39.852308 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:39.852308 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:39.852308 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:39.853196 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:35:39.853196 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:35:39.853196 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:35:39.853196 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:35:39.853196 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:35:39.853196 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:35:39.853196 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:35:39.853196 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:39.857190 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 348/697] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:35:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:39.858364 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f614248d-4056-4247-a0fc-599a8a0edc53 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:39.919802 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ea200-fa65-443c-a8b5-c5c0d96980ba None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:39.954419 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:39.960571 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96455662-0fca-46e6-bc39-2b3430cb0137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:39.961430 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96455662-0fca-46e6-bc39-2b3430cb0137 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:39.961721 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96455662-0fca-46e6-bc39-2b3430cb0137 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:39.974920 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96455662-0fca-46e6-bc39-2b3430cb0137 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:39.985855 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96455662-0fca-46e6-bc39-2b3430cb0137 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:39.992311 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96455662-0fca-46e6-bc39-2b3430cb0137 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:40.014243 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f614248d-4056-4247-a0fc-599a8a0edc53 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:40.027216 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-96455662-0fca-46e6-bc39-2b3430cb0137 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:40.027216 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:40.027216 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:40.027216 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:40.027216 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:40.027216 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:40.027216 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:40.027216 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:40.027216 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:40.027216 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:40.027216 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:40.027216 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:40.027216 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:40.027216 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:40.027216 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:40.028136 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:40.028136 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:40.028136 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:40.028136 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:40.028136 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:40.028136 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:40.028136 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:40.028136 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:40.028136 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:40.028136 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:40.028136 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:40.028136 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:40.028136 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:40.028136 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:40.028136 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:40.028136 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:40.028136 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:40.029028 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:40.029028 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:40.029028 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:40.029028 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:40.035304 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 350/698] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:40.040910 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96455662-0fca-46e6-bc39-2b3430cb0137 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:40.050097 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fa27ec-b78d-4cb4-bce0-cf2bab5b3e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:40.050849 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa27ec-b78d-4cb4-bce0-cf2bab5b3e6d None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:40.051139 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa27ec-b78d-4cb4-bce0-cf2bab5b3e6d None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:40.051415 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa27ec-b78d-4cb4-bce0-cf2bab5b3e6d None admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:40.052215 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fa27ec-b78d-4cb4-bce0-cf2bab5b3e6d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:40.052524 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fa27ec-b78d-4cb4-bce0-cf2bab5b3e6d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:40.054522 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fa27ec-b78d-4cb4-bce0-cf2bab5b3e6d None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:40.073239 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 349/699] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 22:35:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:35:40.074446 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fa27ec-b78d-4cb4-bce0-cf2bab5b3e6d None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:40.183687 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fa27ec-b78d-4cb4-bce0-cf2bab5b3e6d None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:40.222006 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f4025-297d-4cfe-ad05-ce12792deba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:40.222666 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f4025-297d-4cfe-ad05-ce12792deba7 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:40.222982 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f4025-297d-4cfe-ad05-ce12792deba7 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:40.223270 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f4025-297d-4cfe-ad05-ce12792deba7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:40.224110 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f4025-297d-4cfe-ad05-ce12792deba7 None admin] RBAC: Adding query filter params (role.id=715d866890d14923b11470d544b4f960, scope.project.id=aa11ab8229b642ce962193b74001d3bc, user.id=ba7eef4e1b27476ab4016449d9c3a4b9) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:40.224427 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f4025-297d-4cfe-ad05-ce12792deba7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:40.226382 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f4025-297d-4cfe-ad05-ce12792deba7 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:40.230123 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:40.239216 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 350/700] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 22:35:40 2022] GET /identity/v3/role_assignments?user.id=ba7eef4e1b27476ab4016449d9c3a4b9&scope.project.id=aa11ab8229b642ce962193b74001d3bc&role.id=715d866890d14923b11470d544b4f960 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:35:40.259227 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:40.358882 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96455662-0fca-46e6-bc39-2b3430cb0137 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:40.411891 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c69a7d-db1e-4360-96c2-94c68d6a42a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:40.412670 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c69a7d-db1e-4360-96c2-94c68d6a42a1 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:40.412954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c69a7d-db1e-4360-96c2-94c68d6a42a1 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:40.413232 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c69a7d-db1e-4360-96c2-94c68d6a42a1 None admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:40.413968 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c69a7d-db1e-4360-96c2-94c68d6a42a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:40.414275 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c69a7d-db1e-4360-96c2-94c68d6a42a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:40.415720 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c69a7d-db1e-4360-96c2-94c68d6a42a1 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:40.434331 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 351/701] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:40.440330 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f4025-297d-4cfe-ad05-ce12792deba7 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:40.557026 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f4025-297d-4cfe-ad05-ce12792deba7 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:40.594196 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d6e4a-8096-4579-b96c-7bad1d35cc0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:40.595033 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d6e4a-8096-4579-b96c-7bad1d35cc0f None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:40.595318 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d6e4a-8096-4579-b96c-7bad1d35cc0f None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:40.595731 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d6e4a-8096-4579-b96c-7bad1d35cc0f None admin] PATH_INFO: `/v3/users/nova` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:40.606389 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4d6e4a-8096-4579-b96c-7bad1d35cc0f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:40.607915 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4d6e4a-8096-4579-b96c-7bad1d35cc0f None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:40.624075 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-da4d6e4a-8096-4579-b96c-7bad1d35cc0f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:35:40.624075 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:40.624075 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:40.624075 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:40.624075 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:40.624075 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:40.624075 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:40.624075 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:40.624075 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:40.624075 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:40.624075 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:40.624075 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:40.624075 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:35:40.624075 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:35:40.624075 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:35:40.624075 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:35:40.625021 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:40.625021 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:40.625021 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:35:40.625021 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:35:40.625021 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:35:40.625021 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:35:40.625021 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:40.625021 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:40.625021 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:40.625021 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:40.625021 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:40.625021 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:40.625021 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:40.625021 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:40.625021 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:40.625021 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:40.625021 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:40.625021 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:40.626630 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:40.626630 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:40.626630 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:35:40.626630 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:35:40.626630 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:35:40.626630 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:35:40.626630 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:35:40.626630 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:35:40.626630 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:35:40.626630 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:40.626630 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 351/702] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Jul 21 22:35:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:40.631160 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c69a7d-db1e-4360-96c2-94c68d6a42a1 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:40.640090 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ec34a-8d40-438c-af18-7ffe363af342 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:40.640379 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ec34a-8d40-438c-af18-7ffe363af342 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:40.640659 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ec34a-8d40-438c-af18-7ffe363af342 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:40.642298 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 352/703] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:35:40.648351 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a0dbc-0624-4aca-b71e-17b9bec1dc9e None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:40.648721 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a0dbc-0624-4aca-b71e-17b9bec1dc9e None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:40.648892 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a0dbc-0624-4aca-b71e-17b9bec1dc9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:40.669511 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-f65a0dbc-0624-4aca-b71e-17b9bec1dc9e None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:40.671434 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-f65a0dbc-0624-4aca-b71e-17b9bec1dc9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:40.710848 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:40.767677 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 353/704] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:35:40.769355 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e296afe-c4e4-4ba1-8b5e-ec5a03c536c0 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:40.769355 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e296afe-c4e4-4ba1-8b5e-ec5a03c536c0 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:40.769629 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e296afe-c4e4-4ba1-8b5e-ec5a03c536c0 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:40.772290 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c69a7d-db1e-4360-96c2-94c68d6a42a1 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:40.774916 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 354/705] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:40.776625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa4fee-9dea-40eb-bf1b-f3d9f6da5ae8 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:40.776916 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa4fee-9dea-40eb-bf1b-f3d9f6da5ae8 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:40.777191 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa4fee-9dea-40eb-bf1b-f3d9f6da5ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:40.784562 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:40.803968 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-6afa4fee-9dea-40eb-bf1b-f3d9f6da5ae8 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:40.805863 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-6afa4fee-9dea-40eb-bf1b-f3d9f6da5ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:40.811488 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80113b3-15a7-41cf-b890-a7c11a1f2735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:40.812223 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80113b3-15a7-41cf-b890-a7c11a1f2735 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:40.812510 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80113b3-15a7-41cf-b890-a7c11a1f2735 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:40.812784 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80113b3-15a7-41cf-b890-a7c11a1f2735 None admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:40.813685 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80113b3-15a7-41cf-b890-a7c11a1f2735 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:40.813994 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80113b3-15a7-41cf-b890-a7c11a1f2735 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:40.815514 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80113b3-15a7-41cf-b890-a7c11a1f2735 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:40.837097 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 352/706] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Jul 21 22:35:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:40.838328 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80113b3-15a7-41cf-b890-a7c11a1f2735 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:40.890910 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 355/707] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 22:35:40.890910 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e2cee8-5436-4337-85d5-bfaebaefdf4d None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:40.890910 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e2cee8-5436-4337-85d5-bfaebaefdf4d None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:40.890910 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e2cee8-5436-4337-85d5-bfaebaefdf4d None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:40.890910 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 356/708] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:40.890910 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e2cee8-5436-4337-85d5-bfaebaefdf4d None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:40.956978 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80113b3-15a7-41cf-b890-a7c11a1f2735 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:40.994279 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0c3b4b-4f17-4626-8f05-db235447b087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:40.995127 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c3b4b-4f17-4626-8f05-db235447b087 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:40.995479 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:40.996676 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c3b4b-4f17-4626-8f05-db235447b087 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:40.996800 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c3b4b-4f17-4626-8f05-db235447b087 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:40.997417 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0c3b4b-4f17-4626-8f05-db235447b087 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:40.998811 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0c3b4b-4f17-4626-8f05-db235447b087 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:41.006350 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e2cee8-5436-4337-85d5-bfaebaefdf4d None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:41.014225 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-fb0c3b4b-4f17-4626-8f05-db235447b087 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 22:35:41.014225 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:41.014225 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:35:41.014225 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:35:41.014225 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:35:41.014225 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:35:41.014225 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:35:41.014225 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:35:41.014225 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 22:35:41.014225 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:41.014225 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:35:41.014225 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:41.014225 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:41.014225 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:41.014225 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:41.014225 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:41.015239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:41.015239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:41.015239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:41.015239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:41.015239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:41.015239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:41.015239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:41.015239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:35:41.015239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:35:41.015239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:35:41.015239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:35:41.015239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:41.015239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:41.015239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:41.015239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:41.015239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:41.016293 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:41.016293 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:41.016293 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:41.016293 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:41.016293 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:41.016293 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:41.016293 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:41.016293 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:41.016293 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:41.016293 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:41.016293 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:41.016293 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:35:41.016293 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:35:41.016293 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 22:35:41.016293 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:41.016293 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 353/709] 10.222.0.52 () {56 vars in 1129 bytes} [Thu Jul 21 22:35:40 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:41.017738 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2352bccb-31a5-4699-980c-e7b077ae5b93 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:41.017738 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2352bccb-31a5-4699-980c-e7b077ae5b93 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:41.017738 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2352bccb-31a5-4699-980c-e7b077ae5b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:41.041094 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-2352bccb-31a5-4699-980c-e7b077ae5b93 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:41.042288 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-2352bccb-31a5-4699-980c-e7b077ae5b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:41.046635 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e9c838-8ed1-45b1-a2fd-654b3aa1d5d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:41.047326 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e9c838-8ed1-45b1-a2fd-654b3aa1d5d5 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:41.047605 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e9c838-8ed1-45b1-a2fd-654b3aa1d5d5 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:41.047878 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e9c838-8ed1-45b1-a2fd-654b3aa1d5d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:41.069659 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:41.119141 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 354/710] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:35:41.120121 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2352bccb-31a5-4699-980c-e7b077ae5b93 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:41.132013 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e9c838-8ed1-45b1-a2fd-654b3aa1d5d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:41.134462 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e9c838-8ed1-45b1-a2fd-654b3aa1d5d5 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:41.225865 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2352bccb-31a5-4699-980c-e7b077ae5b93 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:41.237347 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:41.253160 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 357/711] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:35:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:35:41.253902 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e9c838-8ed1-45b1-a2fd-654b3aa1d5d5 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:41.261966 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948e2ce8-2d66-468e-a819-4a722d81312f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:41.262786 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e2ce8-2d66-468e-a819-4a722d81312f None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:41.263072 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e2ce8-2d66-468e-a819-4a722d81312f None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:41.263342 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e2ce8-2d66-468e-a819-4a722d81312f None admin] PATH_INFO: `/v3/services/network` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:41.264311 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948e2ce8-2d66-468e-a819-4a722d81312f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:41.265680 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948e2ce8-2d66-468e-a819-4a722d81312f None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:41.277858 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-948e2ce8-2d66-468e-a819-4a722d81312f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 22:35:41.277858 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:41.277858 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:35:41.277858 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:35:41.277858 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:35:41.277858 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:35:41.277858 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:35:41.277858 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:35:41.277858 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 22:35:41.277858 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:41.277858 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:35:41.277858 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:41.277858 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:41.277858 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:41.277858 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:41.277858 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:41.277858 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:41.278882 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:41.278882 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:41.278882 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:41.278882 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:41.278882 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:41.278882 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:41.278882 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:35:41.278882 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:35:41.278882 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:35:41.278882 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:35:41.278882 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:41.278882 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:41.278882 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:41.278882 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:41.278882 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:41.278882 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:41.279727 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:41.279727 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:41.279727 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:41.279727 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:41.279727 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:41.279727 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:41.279727 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:41.279727 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:41.279727 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:41.279727 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:41.279727 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:35:41.279727 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:35:41.279727 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 22:35:41.279727 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:41.279727 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 355/712] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:35:41 2022] GET /identity/v3/services/network => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:41.280510 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948e2ce8-2d66-468e-a819-4a722d81312f None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:41.309652 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:41.368411 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e9c838-8ed1-45b1-a2fd-654b3aa1d5d5 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:41.389062 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948e2ce8-2d66-468e-a819-4a722d81312f None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:41.403563 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44548ef0-d181-4f16-a325-761414978c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:41.404225 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44548ef0-d181-4f16-a325-761414978c05 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:41.404504 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44548ef0-d181-4f16-a325-761414978c05 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:41.404775 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44548ef0-d181-4f16-a325-761414978c05 None admin] PATH_INFO: `/v3/services` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:41.405472 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44548ef0-d181-4f16-a325-761414978c05 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:41.405774 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44548ef0-d181-4f16-a325-761414978c05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:41.407154 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44548ef0-d181-4f16-a325-761414978c05 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:41.415828 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 358/713] 10.222.0.52 () {56 vars in 1124 bytes} [Thu Jul 21 22:35:41 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:41.415828 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44548ef0-d181-4f16-a325-761414978c05 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:41.428551 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f95dc2d-ddcd-44ec-9149-b38c893bbf39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:41.429354 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95dc2d-ddcd-44ec-9149-b38c893bbf39 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:41.429640 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95dc2d-ddcd-44ec-9149-b38c893bbf39 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:41.429919 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95dc2d-ddcd-44ec-9149-b38c893bbf39 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:41.440504 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f95dc2d-ddcd-44ec-9149-b38c893bbf39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:35:41.448926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-2f95dc2d-ddcd-44ec-9149-b38c893bbf39 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:35:41.448926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:41.448926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:41.448926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:41.448926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:41.448926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:41.448926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:41.448926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:41.448926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:41.448926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:41.448926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:41.448926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:41.448926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:35:41.448926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:35:41.448926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:35:41.448926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:35:41.449887 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:35:41.449887 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:35:41.449887 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:35:41.449887 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:35:41.449887 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:41.449887 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:41.449887 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:41.449887 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:41.449887 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:41.449887 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:41.449887 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:41.449887 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:41.449887 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:41.449887 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:41.449887 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:41.449887 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:41.456056 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:41.456056 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:41.456056 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:41.456056 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:41.456056 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:35:41.456056 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:35:41.456056 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:35:41.456056 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:35:41.456056 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:35:41.456056 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:35:41.456056 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:35:41.456056 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:41.456056 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 356/714] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 21 22:35:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:41.456056 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f95dc2d-ddcd-44ec-9149-b38c893bbf39 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:41.481809 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:41.534093 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44548ef0-d181-4f16-a325-761414978c05 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:41.554816 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:41.570091 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce9f1d3-7632-4cdf-bc97-a067f2eeb334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:41.570926 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9f1d3-7632-4cdf-bc97-a067f2eeb334 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:41.571252 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9f1d3-7632-4cdf-bc97-a067f2eeb334 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:41.571761 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9f1d3-7632-4cdf-bc97-a067f2eeb334 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:41.572214 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f95dc2d-ddcd-44ec-9149-b38c893bbf39 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:41.584383 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce9f1d3-7632-4cdf-bc97-a067f2eeb334 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:41.585938 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce9f1d3-7632-4cdf-bc97-a067f2eeb334 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:41.604373 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-fce9f1d3-7632-4cdf-bc97-a067f2eeb334 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:41.604373 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:41.604373 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:41.604373 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:41.604373 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:41.604373 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:41.604373 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:41.604373 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:41.604373 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:41.604373 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:41.604373 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:41.604373 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:41.604373 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:41.604373 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:41.604373 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:41.605298 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:41.605298 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:41.605298 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:41.605298 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:41.605298 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:41.605298 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:41.605298 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:41.605298 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:41.605298 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:41.605298 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:41.605298 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:41.605298 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:41.605298 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:41.605298 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:41.605298 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:41.605298 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:41.605298 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:41.606221 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:41.606221 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:41.606221 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:41.606221 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:41.606221 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 359/715] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:41.606506 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce9f1d3-7632-4cdf-bc97-a067f2eeb334 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:41.614691 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede3a42e-e26a-46a9-80f1-010c4a35f7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:41.615975 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3a42e-e26a-46a9-80f1-010c4a35f7b7 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:41.616268 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3a42e-e26a-46a9-80f1-010c4a35f7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:41.616545 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3a42e-e26a-46a9-80f1-010c4a35f7b7 None admin] PATH_INFO: `/v3/services` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:41.617257 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede3a42e-e26a-46a9-80f1-010c4a35f7b7 None admin] RBAC: Adding query filter params (name=network) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:41.617564 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede3a42e-e26a-46a9-80f1-010c4a35f7b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:41.618947 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede3a42e-e26a-46a9-80f1-010c4a35f7b7 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:41.623655 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 357/716] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Jul 21 22:35:41 2022] GET /identity/v3/services?name=network => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:41.628833 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede3a42e-e26a-46a9-80f1-010c4a35f7b7 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:41.715774 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce9f1d3-7632-4cdf-bc97-a067f2eeb334 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:41.723692 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:41.742916 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede3a42e-e26a-46a9-80f1-010c4a35f7b7 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:41.755737 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e82597-604c-4e1a-aeef-e4e2757a3ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:41.756413 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e82597-604c-4e1a-aeef-e4e2757a3ad6 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:41.756751 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e82597-604c-4e1a-aeef-e4e2757a3ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:41.757026 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e82597-604c-4e1a-aeef-e4e2757a3ad6 None admin] PATH_INFO: `/v3/services` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:41.757743 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e82597-604c-4e1a-aeef-e4e2757a3ad6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:41.758060 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e82597-604c-4e1a-aeef-e4e2757a3ad6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:41.759483 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e82597-604c-4e1a-aeef-e4e2757a3ad6 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:41.766120 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 360/717] 10.222.0.52 () {56 vars in 1124 bytes} [Thu Jul 21 22:35:41 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:41.766945 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e82597-604c-4e1a-aeef-e4e2757a3ad6 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:41.782153 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18627ee-959c-4f7f-a44b-4a2db0447b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:41.783363 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18627ee-959c-4f7f-a44b-4a2db0447b58 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:41.783653 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18627ee-959c-4f7f-a44b-4a2db0447b58 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:41.783937 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18627ee-959c-4f7f-a44b-4a2db0447b58 None admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:41.784716 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18627ee-959c-4f7f-a44b-4a2db0447b58 None admin] RBAC: Adding query filter params (name=service) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:41.785029 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18627ee-959c-4f7f-a44b-4a2db0447b58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:41.786554 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18627ee-959c-4f7f-a44b-4a2db0447b58 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:41.801596 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 358/718] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 21 22:35:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:41.801596 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18627ee-959c-4f7f-a44b-4a2db0447b58 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:41.815386 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:41.922466 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18627ee-959c-4f7f-a44b-4a2db0447b58 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:41.934764 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e82597-604c-4e1a-aeef-e4e2757a3ad6 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:41.977388 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf738f4-1ad9-4057-b47c-a918f9444a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:41.978215 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf738f4-1ad9-4057-b47c-a918f9444a60 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:41.978499 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf738f4-1ad9-4057-b47c-a918f9444a60 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:41.978791 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf738f4-1ad9-4057-b47c-a918f9444a60 None admin] PATH_INFO: `/v3/services` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:41.978992 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e21e8-e261-4d4d-ae41-a2cec640386a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:41.979495 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf738f4-1ad9-4057-b47c-a918f9444a60 None admin] RBAC: Adding query filter params (type=network) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:41.979666 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e21e8-e261-4d4d-ae41-a2cec640386a None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:41.979826 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf738f4-1ad9-4057-b47c-a918f9444a60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:41.979975 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e21e8-e261-4d4d-ae41-a2cec640386a None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:41.980214 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e21e8-e261-4d4d-ae41-a2cec640386a None admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:41.980946 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033e21e8-e261-4d4d-ae41-a2cec640386a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:41.981164 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf738f4-1ad9-4057-b47c-a918f9444a60 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:41.981276 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033e21e8-e261-4d4d-ae41-a2cec640386a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:41.982713 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033e21e8-e261-4d4d-ae41-a2cec640386a None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:41.995063 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 359/719] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Jul 21 22:35:41 2022] GET /identity/v3/services?type=network => generated 124 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:41.996011 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf738f4-1ad9-4057-b47c-a918f9444a60 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:41.996143 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 361/720] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:42.004687 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e21e8-e261-4d4d-ae41-a2cec640386a None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:42.040152 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:42.127188 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:42.144815 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e21e8-e261-4d4d-ae41-a2cec640386a None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:42.156975 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf738f4-1ad9-4057-b47c-a918f9444a60 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:42.210374 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ae551-a75c-430e-8aa1-27460422ffce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:42.211218 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ae551-a75c-430e-8aa1-27460422ffce None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:42.211502 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ae551-a75c-430e-8aa1-27460422ffce None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:42.211772 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ae551-a75c-430e-8aa1-27460422ffce None admin] PATH_INFO: `/v3/projects/service` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:42.215660 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08001208-62af-44c7-b00c-64715eea84ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:42.216467 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08001208-62af-44c7-b00c-64715eea84ad None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:42.216746 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08001208-62af-44c7-b00c-64715eea84ad None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:42.217017 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08001208-62af-44c7-b00c-64715eea84ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:42.228795 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ae551-a75c-430e-8aa1-27460422ffce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:42.230646 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ae551-a75c-430e-8aa1-27460422ffce None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:42.246385 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08001208-62af-44c7-b00c-64715eea84ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:42.247843 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08001208-62af-44c7-b00c-64715eea84ad None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:42.252088 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-a37ae551-a75c-430e-8aa1-27460422ffce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:35:42.252088 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:42.252088 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:42.252088 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:42.252088 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:42.252088 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:42.252088 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:42.252088 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:42.252088 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:42.252088 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:42.252088 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:42.252088 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:42.252088 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:35:42.252088 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:35:42.252088 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:35:42.253331 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:35:42.253331 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:42.253331 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:42.253331 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:42.253331 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:42.253331 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:42.253331 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:42.253331 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:42.253331 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:42.253331 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:42.253331 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:42.253331 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:42.253331 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:42.253331 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:42.253331 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:42.253331 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:42.253331 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:42.254227 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:35:42.254227 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:35:42.254227 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:35:42.254227 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:35:42.254227 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:35:42.254227 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:35:42.254227 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:35:42.254227 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:42.254681 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 362/721] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:35:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:35:42.262417 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ae551-a75c-430e-8aa1-27460422ffce None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:42.277910 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-08001208-62af-44c7-b00c-64715eea84ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:42.277910 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:42.277910 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:42.277910 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:42.277910 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:42.277910 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:42.277910 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:42.277910 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:42.277910 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:42.277910 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:42.277910 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:42.277910 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:42.277910 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:42.277910 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:42.277910 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:42.279255 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:42.279255 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:42.279255 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:42.279255 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:42.279255 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:42.279255 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:42.279255 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:42.279255 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:42.279255 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:42.279255 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:42.279255 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:42.279255 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:42.279255 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:42.279255 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:42.279255 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:42.279255 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:42.279255 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:42.280162 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:42.280162 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:42.280162 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:42.280162 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:42.280162 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 360/722] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:42.285717 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08001208-62af-44c7-b00c-64715eea84ad None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:42.379584 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:42.391725 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ae551-a75c-430e-8aa1-27460422ffce None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:42.437610 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7e9f1f-725e-43c1-9aff-92937af0c0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:42.438290 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e9f1f-725e-43c1-9aff-92937af0c0de None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:42.438573 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e9f1f-725e-43c1-9aff-92937af0c0de None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:42.438872 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e9f1f-725e-43c1-9aff-92937af0c0de None admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:42.439667 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7e9f1f-725e-43c1-9aff-92937af0c0de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:42.439976 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7e9f1f-725e-43c1-9aff-92937af0c0de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:42.441415 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7e9f1f-725e-43c1-9aff-92937af0c0de None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:42.471666 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 363/723] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 22:35:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:42.479714 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7e9f1f-725e-43c1-9aff-92937af0c0de None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:42.512853 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08001208-62af-44c7-b00c-64715eea84ad None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:42.563326 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:42.580556 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6801add-921f-4c58-97cb-074fe1a31c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:42.581237 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6801add-921f-4c58-97cb-074fe1a31c44 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:42.581540 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6801add-921f-4c58-97cb-074fe1a31c44 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:42.581812 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6801add-921f-4c58-97cb-074fe1a31c44 None admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:42.582541 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6801add-921f-4c58-97cb-074fe1a31c44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:42.582875 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6801add-921f-4c58-97cb-074fe1a31c44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:42.592602 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6801add-921f-4c58-97cb-074fe1a31c44 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:42.624215 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 361/724] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:42.639931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6801add-921f-4c58-97cb-074fe1a31c44 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:42.683732 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7e9f1f-725e-43c1-9aff-92937af0c0de None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:42.743787 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c315407-9ee5-4b5b-bd34-0c93aabd8e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:42.744726 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c315407-9ee5-4b5b-bd34-0c93aabd8e1a None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:42.745015 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c315407-9ee5-4b5b-bd34-0c93aabd8e1a None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:42.745294 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c315407-9ee5-4b5b-bd34-0c93aabd8e1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:42.746388 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c315407-9ee5-4b5b-bd34-0c93aabd8e1a None admin] RBAC: Adding query filter params (role.id=d1ddee590a964ff1bbb4ee8716173021, scope.project.id=aa11ab8229b642ce962193b74001d3bc, user.id=6bfdb655a2a44b33ab724daaf73a294b) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:42.747577 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c315407-9ee5-4b5b-bd34-0c93aabd8e1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:42.749014 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c315407-9ee5-4b5b-bd34-0c93aabd8e1a None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:42.757098 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 364/725] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 22:35:42 2022] GET /identity/v3/role_assignments?user.id=6bfdb655a2a44b33ab724daaf73a294b&scope.project.id=aa11ab8229b642ce962193b74001d3bc&role.id=d1ddee590a964ff1bbb4ee8716173021 => generated 259 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:42.758312 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc4dd78-5cb7-4ffc-907d-afd8eb23e123 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:42.758833 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc4dd78-5cb7-4ffc-907d-afd8eb23e123 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:42.759119 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc4dd78-5cb7-4ffc-907d-afd8eb23e123 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:42.761528 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 365/726] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:42.772878 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76735850-1733-470b-9489-cb126180913c None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:42.773239 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76735850-1733-470b-9489-cb126180913c None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:42.773439 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76735850-1733-470b-9489-cb126180913c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:42.786330 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6801add-921f-4c58-97cb-074fe1a31c44 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:42.807709 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-76735850-1733-470b-9489-cb126180913c None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:42.808925 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-76735850-1733-470b-9489-cb126180913c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:42.824614 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dc5d5f-384c-445b-86ca-6daee83d4a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:42.825426 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dc5d5f-384c-445b-86ca-6daee83d4a18 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:42.825704 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dc5d5f-384c-445b-86ca-6daee83d4a18 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:42.825978 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dc5d5f-384c-445b-86ca-6daee83d4a18 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:42.827138 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:42.833786 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dc5d5f-384c-445b-86ca-6daee83d4a18 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:42.835322 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dc5d5f-384c-445b-86ca-6daee83d4a18 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:42.851648 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-e1dc5d5f-384c-445b-86ca-6daee83d4a18 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 22:35:42.851648 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:42.851648 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:42.851648 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:42.851648 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:42.851648 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:42.851648 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:42.851648 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:42.851648 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:42.851648 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:42.851648 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:42.851648 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:42.851648 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:35:42.851648 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:35:42.851648 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:35:42.851648 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:35:42.852895 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:42.852895 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:42.852895 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:35:42.852895 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:35:42.852895 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:35:42.852895 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:35:42.852895 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:42.852895 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:42.852895 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:42.852895 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:42.852895 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:42.852895 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:42.852895 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:42.852895 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:42.852895 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:42.852895 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:42.852895 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:42.852895 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:42.853809 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:42.853809 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:42.853809 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:35:42.853809 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:35:42.853809 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:35:42.853809 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:35:42.853809 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:35:42.853809 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:35:42.853809 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 22:35:42.853809 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:42.857983 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 362/727] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Jul 21 22:35:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:35:42.859164 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dc5d5f-384c-445b-86ca-6daee83d4a18 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:42.929061 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:42.931533 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 366/728] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:35:42.989608 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76735850-1733-470b-9489-cb126180913c None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:43.011922 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dc5d5f-384c-445b-86ca-6daee83d4a18 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:43.051129 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae86eeba-2501-4724-8c8e-057881049d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:43.051815 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae86eeba-2501-4724-8c8e-057881049d32 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:43.052098 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae86eeba-2501-4724-8c8e-057881049d32 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:43.052374 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae86eeba-2501-4724-8c8e-057881049d32 None admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:43.053773 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae86eeba-2501-4724-8c8e-057881049d32 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:43.054085 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae86eeba-2501-4724-8c8e-057881049d32 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:43.055556 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae86eeba-2501-4724-8c8e-057881049d32 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:43.096350 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 363/729] 10.222.0.52 () {56 vars in 1149 bytes} [Thu Jul 21 22:35:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:43.100697 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae86eeba-2501-4724-8c8e-057881049d32 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:43.192913 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76735850-1733-470b-9489-cb126180913c None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:43.258111 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a34e12-0a9f-4414-998c-c9fe02cdcd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:43.258999 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a34e12-0a9f-4414-998c-c9fe02cdcd68 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:43.259286 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a34e12-0a9f-4414-998c-c9fe02cdcd68 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:43.259615 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a34e12-0a9f-4414-998c-c9fe02cdcd68 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:43.276716 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a34e12-0a9f-4414-998c-c9fe02cdcd68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:35:43.279629 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae86eeba-2501-4724-8c8e-057881049d32 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:43.287587 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:43.289175 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-58a34e12-0a9f-4414-998c-c9fe02cdcd68 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:35:43.289175 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:43.289175 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:43.289175 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:43.289175 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:43.289175 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:43.289175 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:43.289175 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:43.289175 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:43.289175 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:43.289175 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:43.289175 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:43.289175 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:35:43.289175 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:35:43.289175 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:35:43.289175 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:35:43.290090 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:35:43.290090 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:35:43.290090 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:35:43.290090 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:35:43.290090 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:43.290090 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:43.290090 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:43.290090 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:43.290090 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:43.290090 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:43.290090 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:43.290090 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:43.290090 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:43.290090 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:43.290090 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:43.290090 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:43.290948 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:43.290948 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:43.290948 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:43.290948 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:43.290948 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:35:43.290948 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:35:43.290948 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:35:43.290948 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:35:43.290948 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:35:43.290948 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:35:43.290948 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:35:43.290948 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:43.295666 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 367/730] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 22:35:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:35:43.296484 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a34e12-0a9f-4414-998c-c9fe02cdcd68 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:43.315143 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:43.341765 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d22de2-0eba-4237-b33f-49918d5b45c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:43.342435 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d22de2-0eba-4237-b33f-49918d5b45c8 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:43.343201 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d22de2-0eba-4237-b33f-49918d5b45c8 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:43.343550 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d22de2-0eba-4237-b33f-49918d5b45c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:43.421762 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d22de2-0eba-4237-b33f-49918d5b45c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:43.423277 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d22de2-0eba-4237-b33f-49918d5b45c8 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:43.535924 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a34e12-0a9f-4414-998c-c9fe02cdcd68 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:43.585704 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf864243-856d-4847-aff1-b4cbd4e2216e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:43.586427 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf864243-856d-4847-aff1-b4cbd4e2216e None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:43.586721 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf864243-856d-4847-aff1-b4cbd4e2216e None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:43.587026 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf864243-856d-4847-aff1-b4cbd4e2216e None admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:43.587778 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf864243-856d-4847-aff1-b4cbd4e2216e None admin] RBAC: Adding query filter params (name=admin) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:43.588091 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf864243-856d-4847-aff1-b4cbd4e2216e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:43.589464 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf864243-856d-4847-aff1-b4cbd4e2216e None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:43.619272 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 368/731] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:35:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:43.623729 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf864243-856d-4847-aff1-b4cbd4e2216e None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:43.661688 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 364/732] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:35:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:35:43.680207 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d22de2-0eba-4237-b33f-49918d5b45c8 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:43.688901 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:43.702716 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:43.742059 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf864243-856d-4847-aff1-b4cbd4e2216e None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:43.780819 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d240cb-ff01-42c1-b623-a057d6b84653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:43.781624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d240cb-ff01-42c1-b623-a057d6b84653 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:43.781912 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d240cb-ff01-42c1-b623-a057d6b84653 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:43.782192 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d240cb-ff01-42c1-b623-a057d6b84653 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:43.794764 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d240cb-ff01-42c1-b623-a057d6b84653 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:43.796513 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d240cb-ff01-42c1-b623-a057d6b84653 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:43.815628 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-a4d240cb-ff01-42c1-b623-a057d6b84653 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:43.815628 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:43.815628 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:43.815628 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:43.815628 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:43.815628 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:43.815628 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:43.815628 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:43.815628 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:43.815628 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:43.815628 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:43.815628 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:43.815628 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:43.815628 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:43.815628 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:43.816530 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:43.816530 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:43.816530 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:43.816530 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:43.816530 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:43.816530 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:43.816530 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:43.816530 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:43.816530 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:43.816530 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:43.816530 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:43.816530 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:43.816530 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:43.816530 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:43.816530 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:43.816530 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:43.816530 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:43.817395 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:43.817395 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:43.817395 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:43.817395 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:43.817395 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 369/733] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:43.824035 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d240cb-ff01-42c1-b623-a057d6b84653 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:43.936039 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d240cb-ff01-42c1-b623-a057d6b84653 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:43.958264 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:43.982883 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78691a6-bea9-4db0-8cc4-09b9915983b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:43.983563 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78691a6-bea9-4db0-8cc4-09b9915983b7 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:43.983847 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78691a6-bea9-4db0-8cc4-09b9915983b7 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:43.984128 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78691a6-bea9-4db0-8cc4-09b9915983b7 None admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:43.984874 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78691a6-bea9-4db0-8cc4-09b9915983b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:43.985182 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78691a6-bea9-4db0-8cc4-09b9915983b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:43.986563 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78691a6-bea9-4db0-8cc4-09b9915983b7 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:43.987717 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d22de2-0eba-4237-b33f-49918d5b45c8 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:44.020696 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 370/734] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:44.026313 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78691a6-bea9-4db0-8cc4-09b9915983b7 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:44.071620 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9212cff3-cde8-4d26-9f84-d24020eada64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:44.072356 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9212cff3-cde8-4d26-9f84-d24020eada64 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:44.072642 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9212cff3-cde8-4d26-9f84-d24020eada64 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:44.072918 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9212cff3-cde8-4d26-9f84-d24020eada64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:44.097236 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9212cff3-cde8-4d26-9f84-d24020eada64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:44.098717 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9212cff3-cde8-4d26-9f84-d24020eada64 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:44.131378 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-9212cff3-cde8-4d26-9f84-d24020eada64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:44.131378 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:44.131378 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:44.131378 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:44.131378 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:44.131378 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:44.131378 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:44.131378 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:44.131378 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:44.131378 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:44.131378 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:44.131378 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:44.131378 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:44.131378 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:44.131378 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:44.132344 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:44.132344 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:44.132344 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:44.132344 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:44.132344 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:44.132344 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:44.132344 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:44.132344 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:44.132344 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:44.132344 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:44.132344 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:44.132344 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:44.132344 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:44.132344 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:44.132344 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:44.132344 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:44.132344 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:44.133241 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:44.133241 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:44.133241 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:44.133241 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:44.133241 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 365/735] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 453 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:44.134810 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31277109-07da-46be-a86a-9bafcb8b58f2 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:44.134810 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31277109-07da-46be-a86a-9bafcb8b58f2 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:44.134992 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31277109-07da-46be-a86a-9bafcb8b58f2 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:44.136472 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 366/736] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:44.141909 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31277109-07da-46be-a86a-9bafcb8b58f2 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:44.209715 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:44.271903 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31277109-07da-46be-a86a-9bafcb8b58f2 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:44.279752 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78691a6-bea9-4db0-8cc4-09b9915983b7 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:44.317525 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572310f9-1e09-424f-8a20-41c068910d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:44.319204 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572310f9-1e09-424f-8a20-41c068910d89 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:44.319663 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572310f9-1e09-424f-8a20-41c068910d89 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:44.320088 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572310f9-1e09-424f-8a20-41c068910d89 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:44.320970 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853e9c29-6c21-405b-8fdf-cb9bc3b5ec21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:44.321689 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e9c29-6c21-405b-8fdf-cb9bc3b5ec21 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:44.321924 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e9c29-6c21-405b-8fdf-cb9bc3b5ec21 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:44.322457 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e9c29-6c21-405b-8fdf-cb9bc3b5ec21 None admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:44.323811 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853e9c29-6c21-405b-8fdf-cb9bc3b5ec21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:44.324141 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853e9c29-6c21-405b-8fdf-cb9bc3b5ec21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:44.325514 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853e9c29-6c21-405b-8fdf-cb9bc3b5ec21 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:44.328578 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572310f9-1e09-424f-8a20-41c068910d89 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:44.330120 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572310f9-1e09-424f-8a20-41c068910d89 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:44.339997 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 367/737] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:44.341427 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268551b-544e-45c6-9967-dae6abc506cb None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:44.341722 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268551b-544e-45c6-9967-dae6abc506cb None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:44.342009 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268551b-544e-45c6-9967-dae6abc506cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:44.346441 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-572310f9-1e09-424f-8a20-41c068910d89 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 22:35:44.346441 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:44.346441 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:44.346441 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:44.346441 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:44.346441 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:44.346441 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:44.346441 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:44.346441 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:44.346441 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:44.346441 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:44.346441 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:44.346441 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:35:44.346441 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:35:44.346441 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:35:44.346441 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:35:44.347896 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:44.347896 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:44.347896 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:35:44.347896 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:35:44.347896 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:35:44.347896 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:35:44.347896 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:44.347896 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:44.347896 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:44.347896 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:44.347896 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:44.347896 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:44.347896 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:44.347896 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:44.347896 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:44.347896 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:44.347896 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:44.347896 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:44.349856 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:44.349856 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:44.349856 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:35:44.349856 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:35:44.349856 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:35:44.349856 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:35:44.349856 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:35:44.349856 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:35:44.349856 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 22:35:44.349856 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:44.354002 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 371/738] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 22:35:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:35:44.354002 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572310f9-1e09-424f-8a20-41c068910d89 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:44.361141 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-2268551b-544e-45c6-9967-dae6abc506cb None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:44.362910 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-2268551b-544e-45c6-9967-dae6abc506cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:44.397185 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:44.459512 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 368/739] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:35:44.460696 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2268551b-544e-45c6-9967-dae6abc506cb None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:44.469060 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572310f9-1e09-424f-8a20-41c068910d89 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:44.509582 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bd81dc-6287-4177-a4b0-8f652bd402a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:44.510093 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd81dc-6287-4177-a4b0-8f652bd402a5 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:44.510378 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd81dc-6287-4177-a4b0-8f652bd402a5 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:44.510654 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd81dc-6287-4177-a4b0-8f652bd402a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:44.517761 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bd81dc-6287-4177-a4b0-8f652bd402a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:44.519227 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bd81dc-6287-4177-a4b0-8f652bd402a5 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:44.523633 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:44.533692 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-78bd81dc-6287-4177-a4b0-8f652bd402a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:35:44.533692 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:44.533692 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:44.533692 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:44.533692 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:44.533692 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:44.533692 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:44.533692 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:44.533692 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:44.533692 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:44.533692 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:44.533692 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:44.533692 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:35:44.533692 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:35:44.533692 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:35:44.534633 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:35:44.534633 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:44.534633 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:44.534633 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:44.534633 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:44.534633 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:44.534633 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:44.534633 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:44.534633 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:44.534633 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:44.534633 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:44.534633 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:44.534633 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:44.534633 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:44.534633 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:44.534633 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:44.534633 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:44.535633 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:35:44.535633 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:35:44.535633 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:35:44.535633 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:35:44.535633 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:35:44.535633 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:35:44.535633 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:35:44.535633 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:44.539178 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 372/740] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:35:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:44.540860 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62981dbb-0634-4673-a9fe-ffb42f1fd5a8 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:44.541330 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62981dbb-0634-4673-a9fe-ffb42f1fd5a8 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:44.541658 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62981dbb-0634-4673-a9fe-ffb42f1fd5a8 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:44.543002 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 373/741] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:44.543674 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62981dbb-0634-4673-a9fe-ffb42f1fd5a8 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:44.582440 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2268551b-544e-45c6-9967-dae6abc506cb None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:44.623683 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a187d85d-19c8-4165-97e7-beb4380c8703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:44.625032 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187d85d-19c8-4165-97e7-beb4380c8703 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:44.625312 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187d85d-19c8-4165-97e7-beb4380c8703 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:44.625652 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187d85d-19c8-4165-97e7-beb4380c8703 None admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:44.626505 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a187d85d-19c8-4165-97e7-beb4380c8703 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:44.627470 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a187d85d-19c8-4165-97e7-beb4380c8703 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:44.629460 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a187d85d-19c8-4165-97e7-beb4380c8703 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:44.650562 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 369/742] 10.222.0.52 () {56 vars in 1135 bytes} [Thu Jul 21 22:35:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:44.651615 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a187d85d-19c8-4165-97e7-beb4380c8703 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:44.659186 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:44.664441 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62981dbb-0634-4673-a9fe-ffb42f1fd5a8 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:44.700609 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074f53de-09a2-408f-b8c2-597103f05a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:44.701281 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074f53de-09a2-408f-b8c2-597103f05a62 None admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:44.701560 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074f53de-09a2-408f-b8c2-597103f05a62 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:44.701839 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074f53de-09a2-408f-b8c2-597103f05a62 None admin] PATH_INFO: `/v3/services` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:44.702564 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074f53de-09a2-408f-b8c2-597103f05a62 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:44.703978 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074f53de-09a2-408f-b8c2-597103f05a62 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:44.757344 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a187d85d-19c8-4165-97e7-beb4380c8703 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:44.760259 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:44.791920 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa584e7-bb2f-451c-ac9c-953fe9d881a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:44.792585 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa584e7-bb2f-451c-ac9c-953fe9d881a8 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:44.792876 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa584e7-bb2f-451c-ac9c-953fe9d881a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:44.793126 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa584e7-bb2f-451c-ac9c-953fe9d881a8 None admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:44.793908 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa584e7-bb2f-451c-ac9c-953fe9d881a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:44.794214 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa584e7-bb2f-451c-ac9c-953fe9d881a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:44.795668 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa584e7-bb2f-451c-ac9c-953fe9d881a8 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:44.807172 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 374/743] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jul 21 22:35:44 2022] POST /identity/v3/services => generated 251 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:44.807172 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a5f0ee-ab9f-4468-8382-db782be0b7be None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:44.807172 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a5f0ee-ab9f-4468-8382-db782be0b7be None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:44.807172 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a5f0ee-ab9f-4468-8382-db782be0b7be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:44.813205 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 370/744] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 22:35:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:44.814051 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa584e7-bb2f-451c-ac9c-953fe9d881a8 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:44.827333 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-19a5f0ee-ab9f-4468-8382-db782be0b7be None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:44.829171 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-19a5f0ee-ab9f-4468-8382-db782be0b7be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:44.928955 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa584e7-bb2f-451c-ac9c-953fe9d881a8 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:44.946799 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 375/745] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:44 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 22:35:44.946799 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a5f0ee-ab9f-4468-8382-db782be0b7be None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:44.989253 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7300fd50-06e5-4e40-98fc-c14c4b2c5e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:44.989915 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300fd50-06e5-4e40-98fc-c14c4b2c5e0b None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:44.990722 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300fd50-06e5-4e40-98fc-c14c4b2c5e0b None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:44.991023 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300fd50-06e5-4e40-98fc-c14c4b2c5e0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:45.026462 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:45.056150 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:45.069786 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7300fd50-06e5-4e40-98fc-c14c4b2c5e0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:45.071283 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7300fd50-06e5-4e40-98fc-c14c4b2c5e0b None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:45.073611 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a5f0ee-ab9f-4468-8382-db782be0b7be None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:45.109461 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be28111a-e02c-454e-9249-eb37ddd40b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:45.110117 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be28111a-e02c-454e-9249-eb37ddd40b59 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:45.110400 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be28111a-e02c-454e-9249-eb37ddd40b59 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:45.110684 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be28111a-e02c-454e-9249-eb37ddd40b59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:45.111588 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be28111a-e02c-454e-9249-eb37ddd40b59 None admin] RBAC: Adding query filter params (role.id=715d866890d14923b11470d544b4f960, scope.project.id=aa11ab8229b642ce962193b74001d3bc, user.id=63cc904578554aea84ff1917a5ed075e) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:45.111899 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be28111a-e02c-454e-9249-eb37ddd40b59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:45.113318 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be28111a-e02c-454e-9249-eb37ddd40b59 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:45.119754 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 376/746] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 22:35:44 2022] GET /identity/v3/role_assignments?user.id=63cc904578554aea84ff1917a5ed075e&scope.project.id=aa11ab8229b642ce962193b74001d3bc&role.id=715d866890d14923b11470d544b4f960 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:45.120853 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be28111a-e02c-454e-9249-eb37ddd40b59 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:45.215253 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 371/747] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:35:44 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 22:35:45.217042 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae12df76-847f-4bd5-a4c7-1950b06b5826 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:45.217548 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae12df76-847f-4bd5-a4c7-1950b06b5826 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:45.218335 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae12df76-847f-4bd5-a4c7-1950b06b5826 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:45.224108 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 372/748] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:45.225363 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae12df76-847f-4bd5-a4c7-1950b06b5826 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:45.251448 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be28111a-e02c-454e-9249-eb37ddd40b59 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:45.297725 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441f2e0-c67a-4c1d-988e-54c3cf3bd8b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:45.299210 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441f2e0-c67a-4c1d-988e-54c3cf3bd8b1 None admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:45.299953 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441f2e0-c67a-4c1d-988e-54c3cf3bd8b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:45.300644 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441f2e0-c67a-4c1d-988e-54c3cf3bd8b1 None admin] PATH_INFO: `/v3/services` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:45.301527 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a441f2e0-c67a-4c1d-988e-54c3cf3bd8b1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:45.303259 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a441f2e0-c67a-4c1d-988e-54c3cf3bd8b1 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:45.313266 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:45.339803 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae12df76-847f-4bd5-a4c7-1950b06b5826 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:45.379270 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed842294-9c92-416d-a206-23a13f810bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:45.380170 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed842294-9c92-416d-a206-23a13f810bb4 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:45.380460 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed842294-9c92-416d-a206-23a13f810bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:45.381379 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed842294-9c92-416d-a206-23a13f810bb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:45.381817 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 377/749] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jul 21 22:35:45 2022] POST /identity/v3/services => generated 240 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:45.383603 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21416ece-766a-4cc0-8518-95a9189b7f3c None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:45.384139 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21416ece-766a-4cc0-8518-95a9189b7f3c None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:45.384989 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21416ece-766a-4cc0-8518-95a9189b7f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:45.394897 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed842294-9c92-416d-a206-23a13f810bb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:45.396937 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed842294-9c92-416d-a206-23a13f810bb4 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:45.408068 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-21416ece-766a-4cc0-8518-95a9189b7f3c None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:45.411198 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-21416ece-766a-4cc0-8518-95a9189b7f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:45.414040 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:45.420824 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-ed842294-9c92-416d-a206-23a13f810bb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:45.420824 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:45.420824 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:45.420824 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:45.420824 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:45.420824 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:45.420824 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:45.420824 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:45.420824 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:45.420824 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:45.420824 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:45.420824 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:45.420824 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:45.420824 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:45.420824 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:45.421756 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:45.421756 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:45.421756 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:45.421756 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:45.421756 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:45.421756 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:45.421756 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:45.421756 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:45.421756 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:45.421756 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:45.421756 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:45.421756 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:45.421756 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:45.421756 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:45.421756 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:45.421756 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:45.421756 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:45.422689 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:45.422689 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:45.422689 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:45.422689 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:45.422689 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 373/750] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:45.429941 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed842294-9c92-416d-a206-23a13f810bb4 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:45.579345 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 378/751] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:35:45.602507 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed842294-9c92-416d-a206-23a13f810bb4 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:45.604538 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21416ece-766a-4cc0-8518-95a9189b7f3c None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:45.621500 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:45.640866 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470fbb6c-be71-405d-acd7-8ee3050ece52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:45.641900 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470fbb6c-be71-405d-acd7-8ee3050ece52 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:45.642384 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470fbb6c-be71-405d-acd7-8ee3050ece52 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:45.642970 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470fbb6c-be71-405d-acd7-8ee3050ece52 None admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:45.643899 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470fbb6c-be71-405d-acd7-8ee3050ece52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:45.644412 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470fbb6c-be71-405d-acd7-8ee3050ece52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:45.645988 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470fbb6c-be71-405d-acd7-8ee3050ece52 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:45.663137 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 374/752] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:35:45.665497 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470fbb6c-be71-405d-acd7-8ee3050ece52 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:45.715327 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21416ece-766a-4cc0-8518-95a9189b7f3c None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:45.751966 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4614c97-af16-41e5-97cb-b35e3026193a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:45.752513 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4614c97-af16-41e5-97cb-b35e3026193a None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:45.752804 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4614c97-af16-41e5-97cb-b35e3026193a None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:45.753082 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4614c97-af16-41e5-97cb-b35e3026193a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:45.755062 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:45.765663 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4614c97-af16-41e5-97cb-b35e3026193a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:35:45.776050 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-b4614c97-af16-41e5-97cb-b35e3026193a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:35:45.776050 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:45.776050 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:45.776050 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:45.776050 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:45.776050 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:45.776050 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:45.776050 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:45.776050 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:45.776050 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:45.776050 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:45.776050 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:45.776050 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:35:45.776050 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:35:45.776050 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:35:45.776050 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:35:45.783760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:35:45.783760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:35:45.783760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:35:45.783760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:35:45.783760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:45.783760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:45.783760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:45.783760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:45.783760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:45.783760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:45.783760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:45.783760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:45.783760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:45.783760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:45.783760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:45.783760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:45.784779 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:45.784779 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:45.784779 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:45.784779 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:45.784779 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:35:45.784779 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:35:45.784779 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:35:45.784779 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:35:45.784779 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:35:45.784779 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:35:45.784779 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:35:45.784779 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:45.784779 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 379/753] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 22:35:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:45.785642 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470fbb6c-be71-405d-acd7-8ee3050ece52 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:45.786157 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4614c97-af16-41e5-97cb-b35e3026193a None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:45.827721 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6211acad-15b3-4e3e-80e6-f4b25809eee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:45.828608 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6211acad-15b3-4e3e-80e6-f4b25809eee3 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:45.828962 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6211acad-15b3-4e3e-80e6-f4b25809eee3 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:45.829305 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6211acad-15b3-4e3e-80e6-f4b25809eee3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:45.837216 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6211acad-15b3-4e3e-80e6-f4b25809eee3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:45.838741 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6211acad-15b3-4e3e-80e6-f4b25809eee3 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:45.853919 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-6211acad-15b3-4e3e-80e6-f4b25809eee3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:35:45.853919 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:45.853919 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:45.853919 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:45.853919 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:45.853919 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:45.853919 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:45.853919 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:45.853919 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:45.853919 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:45.853919 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:45.853919 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:45.853919 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:35:45.853919 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:35:45.853919 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:35:45.855055 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:35:45.855055 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:45.855055 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:45.855055 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:45.855055 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:45.855055 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:45.855055 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:45.855055 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:45.855055 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:45.855055 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:45.855055 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:45.855055 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:45.855055 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:45.855055 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:45.855055 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:45.855055 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:45.855055 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:45.855990 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:35:45.855990 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:35:45.855990 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:35:45.855990 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:35:45.855990 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:35:45.855990 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:35:45.855990 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:35:45.855990 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:45.856890 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 375/754] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:35:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:35:45.863418 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6211acad-15b3-4e3e-80e6-f4b25809eee3 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:45.911879 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:45.917334 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4614c97-af16-41e5-97cb-b35e3026193a None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:45.962801 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fbfb6d-f2c9-4520-b6ca-9db15c0b265e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:45.963714 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fbfb6d-f2c9-4520-b6ca-9db15c0b265e None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:45.964186 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fbfb6d-f2c9-4520-b6ca-9db15c0b265e None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:45.964611 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fbfb6d-f2c9-4520-b6ca-9db15c0b265e None admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:45.966497 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fbfb6d-f2c9-4520-b6ca-9db15c0b265e None admin] RBAC: Adding query filter params (name=admin) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:45.967001 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fbfb6d-f2c9-4520-b6ca-9db15c0b265e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:45.969254 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fbfb6d-f2c9-4520-b6ca-9db15c0b265e None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:45.988226 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 380/755] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:35:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:45.995916 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fbfb6d-f2c9-4520-b6ca-9db15c0b265e None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:46.065419 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6211acad-15b3-4e3e-80e6-f4b25809eee3 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:46.072107 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:46.124999 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fbfb6d-f2c9-4520-b6ca-9db15c0b265e None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:46.150766 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4386ac84-a0a6-418a-972a-23f74ba8022c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:46.151422 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4386ac84-a0a6-418a-972a-23f74ba8022c None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:46.151708 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4386ac84-a0a6-418a-972a-23f74ba8022c None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:46.151984 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4386ac84-a0a6-418a-972a-23f74ba8022c None admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:46.152768 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4386ac84-a0a6-418a-972a-23f74ba8022c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:46.153105 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4386ac84-a0a6-418a-972a-23f74ba8022c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:46.154499 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4386ac84-a0a6-418a-972a-23f74ba8022c None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:46.173484 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b928f50f-c73d-433f-b596-f72660da4ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:46.174530 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928f50f-c73d-433f-b596-f72660da4ed9 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:46.174981 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928f50f-c73d-433f-b596-f72660da4ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:46.175399 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928f50f-c73d-433f-b596-f72660da4ed9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:46.187913 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 376/756] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 22:35:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:46.194163 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4386ac84-a0a6-418a-972a-23f74ba8022c None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:46.194873 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b928f50f-c73d-433f-b596-f72660da4ed9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:46.203111 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b928f50f-c73d-433f-b596-f72660da4ed9 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:46.244039 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-b928f50f-c73d-433f-b596-f72660da4ed9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:46.244039 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:46.244039 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:46.244039 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:46.244039 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:46.244039 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:46.244039 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:46.244039 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:46.244039 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:46.244039 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:46.244039 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:46.244039 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:46.244039 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:46.244039 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:46.244039 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:46.245217 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:46.245217 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:46.245217 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:46.245217 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:46.245217 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:46.245217 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:46.245217 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:46.245217 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:46.245217 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:46.245217 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:46.245217 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:46.245217 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:46.245217 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:46.245217 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:46.245217 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:46.245217 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:46.245217 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:46.246265 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:46.246265 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:46.246265 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:46.246265 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:46.246265 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 381/757] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:46.251176 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b928f50f-c73d-433f-b596-f72660da4ed9 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:46.377650 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:46.387248 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4386ac84-a0a6-418a-972a-23f74ba8022c None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:46.458583 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b928f50f-c73d-433f-b596-f72660da4ed9 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:46.461395 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8069a3-a14e-482c-9ae8-c15c6f4990b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:46.462187 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8069a3-a14e-482c-9ae8-c15c6f4990b3 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:46.463253 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8069a3-a14e-482c-9ae8-c15c6f4990b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:46.463253 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8069a3-a14e-482c-9ae8-c15c6f4990b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:46.463893 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8069a3-a14e-482c-9ae8-c15c6f4990b3 None admin] RBAC: Adding query filter params (role.id=d1ddee590a964ff1bbb4ee8716173021, scope.project.id=aa11ab8229b642ce962193b74001d3bc, user.id=ba7eef4e1b27476ab4016449d9c3a4b9) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:46.464308 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8069a3-a14e-482c-9ae8-c15c6f4990b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:46.465847 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8069a3-a14e-482c-9ae8-c15c6f4990b3 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:46.470436 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:46.480494 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 377/758] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 22:35:46 2022] GET /identity/v3/role_assignments?user.id=ba7eef4e1b27476ab4016449d9c3a4b9&scope.project.id=aa11ab8229b642ce962193b74001d3bc&role.id=d1ddee590a964ff1bbb4ee8716173021 => generated 259 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:35:46.507744 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fb88b0-8711-4dde-95e6-5e1e17d818ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:46.508450 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fb88b0-8711-4dde-95e6-5e1e17d818ee None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:46.508738 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fb88b0-8711-4dde-95e6-5e1e17d818ee None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:46.509015 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fb88b0-8711-4dde-95e6-5e1e17d818ee None admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:46.509766 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fb88b0-8711-4dde-95e6-5e1e17d818ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:46.510076 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fb88b0-8711-4dde-95e6-5e1e17d818ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:46.511533 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fb88b0-8711-4dde-95e6-5e1e17d818ee None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:46.537633 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 382/759] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:46.542981 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8069a3-a14e-482c-9ae8-c15c6f4990b3 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:46.698477 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8069a3-a14e-482c-9ae8-c15c6f4990b3 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:46.733786 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df494b-cbbb-455b-a7c6-3f2cc8b14837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:46.734581 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df494b-cbbb-455b-a7c6-3f2cc8b14837 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:46.734901 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df494b-cbbb-455b-a7c6-3f2cc8b14837 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:46.735177 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df494b-cbbb-455b-a7c6-3f2cc8b14837 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:46.743678 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2df494b-cbbb-455b-a7c6-3f2cc8b14837 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:46.745133 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2df494b-cbbb-455b-a7c6-3f2cc8b14837 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:46.760114 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-b2df494b-cbbb-455b-a7c6-3f2cc8b14837 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:35:46.760114 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:46.760114 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:46.760114 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:46.760114 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:46.760114 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:46.760114 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:46.760114 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:46.760114 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:46.760114 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:46.760114 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:46.760114 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:46.760114 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:35:46.760114 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:35:46.760114 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:35:46.760114 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:35:46.761066 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:46.761066 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:46.761066 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:35:46.761066 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:35:46.761066 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:35:46.761066 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:35:46.761066 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:46.761066 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:46.761066 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:46.761066 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:46.761066 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:46.761066 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:46.761066 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:46.761066 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:46.761066 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:46.761066 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:46.761066 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:46.761066 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:46.762923 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:46.762923 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:46.762923 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:35:46.762923 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:35:46.762923 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:35:46.762923 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:35:46.762923 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:35:46.762923 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:35:46.762923 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:35:46.762923 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:46.762923 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 378/760] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Jul 21 22:35:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:46.767840 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fb88b0-8711-4dde-95e6-5e1e17d818ee None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:46.915007 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fb88b0-8711-4dde-95e6-5e1e17d818ee None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:46.950512 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea85783b-b01f-4afa-b2fc-e0fcf8f360bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:46.951542 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea85783b-b01f-4afa-b2fc-e0fcf8f360bc None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:46.952042 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea85783b-b01f-4afa-b2fc-e0fcf8f360bc None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:46.952509 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea85783b-b01f-4afa-b2fc-e0fcf8f360bc None admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:46.953530 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea85783b-b01f-4afa-b2fc-e0fcf8f360bc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:46.954070 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea85783b-b01f-4afa-b2fc-e0fcf8f360bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:46.955745 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea85783b-b01f-4afa-b2fc-e0fcf8f360bc None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:46.986418 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 383/761] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Jul 21 22:35:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:46.991602 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df494b-cbbb-455b-a7c6-3f2cc8b14837 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:47.026560 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:47.116325 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c369cc5-4225-4703-a79a-bfc94476ccc6 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:47.116958 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c369cc5-4225-4703-a79a-bfc94476ccc6 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:47.117434 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c369cc5-4225-4703-a79a-bfc94476ccc6 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:47.120298 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 384/762] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:47.124670 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedb187d-f642-4828-a614-4e6b22ed6435 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:47.125186 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedb187d-f642-4828-a614-4e6b22ed6435 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:47.125709 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedb187d-f642-4828-a614-4e6b22ed6435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:47.145216 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-dedb187d-f642-4828-a614-4e6b22ed6435 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:47.147871 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-dedb187d-f642-4828-a614-4e6b22ed6435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:47.149944 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df494b-cbbb-455b-a7c6-3f2cc8b14837 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:47.171137 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:47.226984 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 385/763] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:35:47.227167 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b50abbb-b888-408c-99a2-b60e2e02c6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:47.227723 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b50abbb-b888-408c-99a2-b60e2e02c6f5 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:47.228008 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b50abbb-b888-408c-99a2-b60e2e02c6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:47.228283 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b50abbb-b888-408c-99a2-b60e2e02c6f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:47.244645 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedb187d-f642-4828-a614-4e6b22ed6435 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:47.249782 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b50abbb-b888-408c-99a2-b60e2e02c6f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:47.251273 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b50abbb-b888-408c-99a2-b60e2e02c6f5 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:47.280518 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-5b50abbb-b888-408c-99a2-b60e2e02c6f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:47.280518 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:47.280518 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:47.280518 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:47.280518 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:47.280518 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:47.280518 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:47.280518 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:47.280518 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:47.280518 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:47.280518 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:47.280518 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:47.280518 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:47.280518 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:47.280518 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:47.281485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:47.281485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:47.281485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:47.281485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:47.281485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:47.281485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:47.281485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:47.281485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:47.281485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:47.281485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:47.281485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:47.281485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:47.281485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:47.281485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:47.281485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:47.281485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:47.281485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:47.282389 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:47.282389 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:47.282389 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:47.282389 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:47.282389 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 379/764] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:47.288379 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b50abbb-b888-408c-99a2-b60e2e02c6f5 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:47.383117 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:47.408640 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedb187d-f642-4828-a614-4e6b22ed6435 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:47.423207 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b50abbb-b888-408c-99a2-b60e2e02c6f5 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:47.444956 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585c25d7-f9db-4163-a268-066dd4e1ae53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:47.445751 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c25d7-f9db-4163-a268-066dd4e1ae53 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:47.446097 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c25d7-f9db-4163-a268-066dd4e1ae53 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:47.446349 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c25d7-f9db-4163-a268-066dd4e1ae53 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:47.447044 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585c25d7-f9db-4163-a268-066dd4e1ae53 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:47.448537 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585c25d7-f9db-4163-a268-066dd4e1ae53 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:47.460814 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d667a-bcdb-4ebd-bf51-e4338a380752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:47.461766 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d667a-bcdb-4ebd-bf51-e4338a380752 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:47.462241 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d667a-bcdb-4ebd-bf51-e4338a380752 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:47.462717 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d667a-bcdb-4ebd-bf51-e4338a380752 None admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:47.463658 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260d667a-bcdb-4ebd-bf51-e4338a380752 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:47.464080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-585c25d7-f9db-4163-a268-066dd4e1ae53 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 22:35:47.464080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:47.464080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:35:47.464080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:35:47.464080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:35:47.464080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:35:47.464080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:35:47.464080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:35:47.464080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 22:35:47.464080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:47.464080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:35:47.464080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:47.464080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:47.464080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:47.464080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:47.464080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:47.465281 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:47.465281 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:47.465281 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:47.465281 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:47.465281 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:47.465281 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:47.465281 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:47.465281 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:35:47.465281 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:35:47.465281 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:35:47.465281 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:35:47.465281 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:47.465281 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:47.465281 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:47.465281 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:47.465281 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:47.466117 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:47.466117 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:47.466117 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:47.466117 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:47.466117 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:47.466117 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:47.466117 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:47.466117 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:47.466117 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:47.466117 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:47.466117 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:47.466117 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:35:47.466117 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:35:47.466117 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 22:35:47.466117 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:47.466117 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 386/765] 10.222.0.52 () {56 vars in 1129 bytes} [Thu Jul 21 22:35:47 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:35:47.467644 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260d667a-bcdb-4ebd-bf51-e4338a380752 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:47.469211 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260d667a-bcdb-4ebd-bf51-e4338a380752 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:47.471656 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585c25d7-f9db-4163-a268-066dd4e1ae53 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:47.483433 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 380/766] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:47.488345 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d667a-bcdb-4ebd-bf51-e4338a380752 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:47.516746 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:47.621067 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585c25d7-f9db-4163-a268-066dd4e1ae53 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:47.632045 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d667a-bcdb-4ebd-bf51-e4338a380752 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:47.667712 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:47.672897 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7711da35-0962-4d46-9e70-16b959ca17f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:47.673628 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6dc371-952d-42e1-b4f0-418c02cf870f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:47.674245 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7711da35-0962-4d46-9e70-16b959ca17f6 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:47.674407 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6dc371-952d-42e1-b4f0-418c02cf870f None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:47.674742 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6dc371-952d-42e1-b4f0-418c02cf870f None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:47.675075 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7711da35-0962-4d46-9e70-16b959ca17f6 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:47.675176 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6dc371-952d-42e1-b4f0-418c02cf870f None admin] PATH_INFO: `/v3/services` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:47.681647 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7711da35-0962-4d46-9e70-16b959ca17f6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:47.681760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6dc371-952d-42e1-b4f0-418c02cf870f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:47.681760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6dc371-952d-42e1-b4f0-418c02cf870f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:47.681760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6dc371-952d-42e1-b4f0-418c02cf870f None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:47.683339 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7711da35-0962-4d46-9e70-16b959ca17f6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:47.683793 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 387/767] 10.222.0.52 () {56 vars in 1124 bytes} [Thu Jul 21 22:35:47 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:47.685067 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826dea8-3f75-4c8f-aecf-6f4db5d46a4a None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:47.685335 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826dea8-3f75-4c8f-aecf-6f4db5d46a4a None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:47.685611 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826dea8-3f75-4c8f-aecf-6f4db5d46a4a None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:47.689485 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 388/768] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:47.690865 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7711da35-0962-4d46-9e70-16b959ca17f6 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:47.695913 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2826dea8-3f75-4c8f-aecf-6f4db5d46a4a None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:47.711657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-7711da35-0962-4d46-9e70-16b959ca17f6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:35:47.711657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:47.711657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:47.711657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:47.711657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:47.711657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:47.711657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:47.711657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:47.711657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:47.711657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:47.711657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:47.711657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:47.711657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:35:47.711657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:35:47.711657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:35:47.712606 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:35:47.712606 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:47.712606 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:47.712606 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:47.712606 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:47.712606 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:47.712606 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:47.712606 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:47.712606 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:47.712606 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:47.712606 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:47.712606 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:47.712606 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:47.712606 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:47.712606 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:47.712606 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:47.712606 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:47.714109 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:35:47.714109 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:35:47.714109 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:35:47.714109 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:35:47.714109 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:35:47.714109 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:35:47.714109 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:35:47.714109 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:47.714109 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 381/769] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:35:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:47.714592 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048032d-fb32-4c68-b08f-41268fadbe06 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:47.714862 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048032d-fb32-4c68-b08f-41268fadbe06 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:47.715018 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c048032d-fb32-4c68-b08f-41268fadbe06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:47.736240 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-c048032d-fb32-4c68-b08f-41268fadbe06 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:47.737368 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-c048032d-fb32-4c68-b08f-41268fadbe06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:47.795267 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:47.818681 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2826dea8-3f75-4c8f-aecf-6f4db5d46a4a None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:47.830918 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 382/770] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:35:47.832244 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c048032d-fb32-4c68-b08f-41268fadbe06 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:47.861171 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd64bf76-5bdf-4ea4-8e43-56ad3acfea45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:47.861949 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd64bf76-5bdf-4ea4-8e43-56ad3acfea45 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:47.862237 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd64bf76-5bdf-4ea4-8e43-56ad3acfea45 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:47.862517 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd64bf76-5bdf-4ea4-8e43-56ad3acfea45 None admin] PATH_INFO: `/v3/services` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:47.864577 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd64bf76-5bdf-4ea4-8e43-56ad3acfea45 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:47.864577 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd64bf76-5bdf-4ea4-8e43-56ad3acfea45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:47.865631 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd64bf76-5bdf-4ea4-8e43-56ad3acfea45 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:47.872315 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 389/771] 10.222.0.52 () {56 vars in 1124 bytes} [Thu Jul 21 22:35:47 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:47.873285 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd64bf76-5bdf-4ea4-8e43-56ad3acfea45 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:47.973864 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c048032d-fb32-4c68-b08f-41268fadbe06 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:47.998080 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:47.999724 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd64bf76-5bdf-4ea4-8e43-56ad3acfea45 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:48.010531 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c407b83-071f-44e3-bd4c-cdb15677f955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:48.011474 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c407b83-071f-44e3-bd4c-cdb15677f955 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:48.011954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c407b83-071f-44e3-bd4c-cdb15677f955 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:48.012441 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c407b83-071f-44e3-bd4c-cdb15677f955 None admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:48.013413 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c407b83-071f-44e3-bd4c-cdb15677f955 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:48.013913 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c407b83-071f-44e3-bd4c-cdb15677f955 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:48.015558 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c407b83-071f-44e3-bd4c-cdb15677f955 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:48.032909 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 383/772] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 22:35:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:48.034233 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c407b83-071f-44e3-bd4c-cdb15677f955 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:48.053453 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efba0ac7-eaf3-4607-ae82-22c1e7dc0187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:48.054350 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba0ac7-eaf3-4607-ae82-22c1e7dc0187 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:48.054635 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba0ac7-eaf3-4607-ae82-22c1e7dc0187 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:48.055413 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba0ac7-eaf3-4607-ae82-22c1e7dc0187 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:48.069876 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efba0ac7-eaf3-4607-ae82-22c1e7dc0187 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:35:48.080540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-efba0ac7-eaf3-4607-ae82-22c1e7dc0187 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:35:48.080540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:48.080540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:48.080540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:48.080540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:48.080540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:48.080540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:48.080540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:48.080540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:48.080540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:48.080540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:48.080540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:48.080540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:35:48.080540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:35:48.080540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:35:48.080540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:35:48.081466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:35:48.081466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:35:48.081466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:35:48.081466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:35:48.081466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:48.081466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:48.081466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:48.081466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:48.081466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:48.081466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:48.081466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:48.081466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:48.081466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:48.081466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:48.081466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:48.081466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:48.087610 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:48.087610 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:48.087610 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:48.087610 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:48.087610 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:35:48.087610 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:35:48.087610 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:35:48.087610 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:35:48.087610 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:35:48.087610 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:35:48.087610 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:35:48.087610 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:48.087610 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 390/773] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 21 22:35:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:48.087610 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efba0ac7-eaf3-4607-ae82-22c1e7dc0187 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:48.096863 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:48.154927 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c407b83-071f-44e3-bd4c-cdb15677f955 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:48.191654 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91018ac7-6166-4066-bd30-227bcfda245a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:48.192327 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91018ac7-6166-4066-bd30-227bcfda245a None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:48.192613 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91018ac7-6166-4066-bd30-227bcfda245a None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:48.192895 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91018ac7-6166-4066-bd30-227bcfda245a None admin] PATH_INFO: `/v3/endpoints` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:48.193717 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91018ac7-6166-4066-bd30-227bcfda245a None admin] RBAC: Adding query filter params (interface=public, service_id=7744bbae0c3b42bf8fbb445a7978f00d, region_id=RegionOne) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:48.194031 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91018ac7-6166-4066-bd30-227bcfda245a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:48.195436 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91018ac7-6166-4066-bd30-227bcfda245a None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:48.204113 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 384/774] 10.222.0.52 () {56 vars in 1251 bytes} [Thu Jul 21 22:35:48 2022] GET /identity/v3/endpoints?service_id=7744bbae0c3b42bf8fbb445a7978f00d&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:48.204950 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91018ac7-6166-4066-bd30-227bcfda245a None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:48.220741 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efba0ac7-eaf3-4607-ae82-22c1e7dc0187 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:48.252045 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:48.262214 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcc8698-3587-4444-8829-0e5935537a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:48.263060 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcc8698-3587-4444-8829-0e5935537a76 None admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:48.270822 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcc8698-3587-4444-8829-0e5935537a76 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:48.270822 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcc8698-3587-4444-8829-0e5935537a76 None admin] PATH_INFO: `/v3/projects/aa11ab8229b642ce962193b74001d3bc/users/6bfdb655a2a44b33ab724daaf73a294b/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:48.270822 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcc8698-3587-4444-8829-0e5935537a76 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa11ab8229b642ce962193b74001d3bc, user_id=6bfdb655a2a44b33ab724daaf73a294b, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:48.270822 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcc8698-3587-4444-8829-0e5935537a76 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:48.306257 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91018ac7-6166-4066-bd30-227bcfda245a None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:48.340043 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a3699e-e4d3-4397-9f07-10973589e43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:48.340706 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a3699e-e4d3-4397-9f07-10973589e43b None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:48.340987 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a3699e-e4d3-4397-9f07-10973589e43b None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:48.341263 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a3699e-e4d3-4397-9f07-10973589e43b None admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:48.342004 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a3699e-e4d3-4397-9f07-10973589e43b None admin] RBAC: Adding query filter params (name=service) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:48.342345 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a3699e-e4d3-4397-9f07-10973589e43b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:48.343757 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a3699e-e4d3-4397-9f07-10973589e43b None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:48.357551 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 391/775] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 21 22:35:48 2022] PUT /identity/v3/projects/aa11ab8229b642ce962193b74001d3bc/users/6bfdb655a2a44b33ab724daaf73a294b/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:48.357747 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 385/776] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 21 22:35:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:48.357747 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a3699e-e4d3-4397-9f07-10973589e43b None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:48.365941 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e935f-b798-4c03-8140-9547eb66bb52 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:48.365941 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e935f-b798-4c03-8140-9547eb66bb52 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:48.365941 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e935f-b798-4c03-8140-9547eb66bb52 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:48.365941 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 392/777] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:48 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:48.365941 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e935f-b798-4c03-8140-9547eb66bb52 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:48.443642 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:48.494469 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e935f-b798-4c03-8140-9547eb66bb52 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:48.511755 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a3699e-e4d3-4397-9f07-10973589e43b None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:48.523943 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:48.542895 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a82076-7296-46f0-b082-187396e7eade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:48.543700 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a82076-7296-46f0-b082-187396e7eade None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:48.543990 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a82076-7296-46f0-b082-187396e7eade None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:48.544266 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a82076-7296-46f0-b082-187396e7eade None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:48.551195 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa65ba4-5057-4b13-943a-cbc6ae5feb1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:48.551866 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa65ba4-5057-4b13-943a-cbc6ae5feb1e None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:48.552148 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa65ba4-5057-4b13-943a-cbc6ae5feb1e None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:48.552443 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa65ba4-5057-4b13-943a-cbc6ae5feb1e None admin] PATH_INFO: `/v3/services` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:48.553131 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa65ba4-5057-4b13-943a-cbc6ae5feb1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:48.554600 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa65ba4-5057-4b13-943a-cbc6ae5feb1e None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:48.558416 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a82076-7296-46f0-b082-187396e7eade None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:48.559906 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a82076-7296-46f0-b082-187396e7eade None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:48.563316 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 386/778] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Jul 21 22:35:48 2022] GET /identity/v3/services => generated 772 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:48.564549 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9dabe-4e8b-40fc-90c0-85d9d21df801 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:48.564836 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9dabe-4e8b-40fc-90c0-85d9d21df801 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:48.565110 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9dabe-4e8b-40fc-90c0-85d9d21df801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:48.583063 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-74a82076-7296-46f0-b082-187396e7eade None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:48.583063 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:48.583063 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:48.583063 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:48.583063 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:48.583063 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:48.583063 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:48.583063 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:48.583063 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:48.583063 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:48.583063 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:48.583063 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:48.583063 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:48.583063 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:48.583063 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:48.584888 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:48.584888 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:48.584888 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:48.584888 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:48.584888 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:48.584888 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:48.584888 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:48.584888 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:48.584888 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:48.584888 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:48.584888 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:48.584888 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:48.584888 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:48.584888 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:48.584888 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:48.584888 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:48.584888 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:48.585768 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:48.585768 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:48.585768 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:48.585768 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:48.588228 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 393/779] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:48.595192 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a82076-7296-46f0-b082-187396e7eade None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:48.598146 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-0dd9dabe-4e8b-40fc-90c0-85d9d21df801 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:48.599437 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-0dd9dabe-4e8b-40fc-90c0-85d9d21df801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:48.680646 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 387/780] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:35:48.715804 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd9dabe-4e8b-40fc-90c0-85d9d21df801 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:48.720749 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a82076-7296-46f0-b082-187396e7eade None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:48.744406 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:48.761223 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844237a3-c021-43af-8b06-64a0b79a26a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:48.762282 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844237a3-c021-43af-8b06-64a0b79a26a1 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:48.762752 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844237a3-c021-43af-8b06-64a0b79a26a1 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:48.763222 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844237a3-c021-43af-8b06-64a0b79a26a1 None admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:48.764128 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844237a3-c021-43af-8b06-64a0b79a26a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:48.764673 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844237a3-c021-43af-8b06-64a0b79a26a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:48.766281 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844237a3-c021-43af-8b06-64a0b79a26a1 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:48.779414 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 394/781] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:35:48.781137 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902483db-0414-4220-9482-1af59301747e None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:48.781621 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902483db-0414-4220-9482-1af59301747e None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:48.782111 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902483db-0414-4220-9482-1af59301747e None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:48.783893 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 395/782] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:48.784842 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902483db-0414-4220-9482-1af59301747e None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:48.862380 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd9dabe-4e8b-40fc-90c0-85d9d21df801 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:48.864485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:48.909321 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428af67-e76f-464e-a68a-a26fcfc0453a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:48.910397 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428af67-e76f-464e-a68a-a26fcfc0453a None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:48.910687 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428af67-e76f-464e-a68a-a26fcfc0453a None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:48.910993 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428af67-e76f-464e-a68a-a26fcfc0453a None admin] PATH_INFO: `/v3/services/network` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:48.911726 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8428af67-e76f-464e-a68a-a26fcfc0453a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:48.913820 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8428af67-e76f-464e-a68a-a26fcfc0453a None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:48.917293 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902483db-0414-4220-9482-1af59301747e None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:48.928823 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-8428af67-e76f-464e-a68a-a26fcfc0453a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 22:35:48.928823 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:48.928823 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:35:48.928823 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:35:48.928823 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:35:48.928823 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:35:48.928823 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:35:48.928823 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:35:48.928823 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 22:35:48.928823 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:48.928823 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:35:48.928823 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:48.928823 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:48.928823 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:48.928823 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:48.928823 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:48.928823 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:48.929846 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:48.929846 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:48.929846 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:48.929846 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:48.929846 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:48.929846 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:48.929846 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:35:48.929846 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:35:48.929846 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:35:48.929846 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:35:48.929846 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:48.929846 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:48.929846 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:48.929846 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:48.929846 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:48.929846 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:48.930664 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:48.930664 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:48.930664 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:48.930664 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:48.930664 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:48.930664 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:48.930664 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:48.930664 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:48.930664 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:48.930664 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:48.930664 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:35:48.930664 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:35:48.930664 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 22:35:48.930664 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:48.934844 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 388/783] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:35:48 2022] GET /identity/v3/services/network => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:35:48.936378 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02af6a8-d600-400d-a95b-9a96a5941b12 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:48.936667 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02af6a8-d600-400d-a95b-9a96a5941b12 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:48.936940 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02af6a8-d600-400d-a95b-9a96a5941b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:48.960446 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-f02af6a8-d600-400d-a95b-9a96a5941b12 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:48.962297 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-f02af6a8-d600-400d-a95b-9a96a5941b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:48.964190 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7ed6c2-c4f3-4612-aa29-bcc7c0d289b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:48.965007 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7ed6c2-c4f3-4612-aa29-bcc7c0d289b8 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:48.965284 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7ed6c2-c4f3-4612-aa29-bcc7c0d289b8 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:48.965610 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7ed6c2-c4f3-4612-aa29-bcc7c0d289b8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:48.974222 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7ed6c2-c4f3-4612-aa29-bcc7c0d289b8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:48.975767 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7ed6c2-c4f3-4612-aa29-bcc7c0d289b8 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:48.992757 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-8b7ed6c2-c4f3-4612-aa29-bcc7c0d289b8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 22:35:48.992757 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:48.992757 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:48.992757 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:48.992757 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:48.992757 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:48.992757 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:48.992757 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:48.992757 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:48.992757 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:48.992757 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:48.992757 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:48.992757 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:35:48.992757 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:35:48.992757 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:35:48.992757 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:35:48.993750 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:48.993750 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:48.993750 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:35:48.993750 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:35:48.993750 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:35:48.993750 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:35:48.993750 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:48.993750 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:48.993750 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:48.993750 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:48.993750 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:48.993750 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:48.993750 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:48.993750 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:48.993750 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:48.993750 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:48.993750 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:48.993750 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:48.994693 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:48.994693 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:48.994693 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:35:48.994693 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:35:48.994693 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:35:48.994693 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:35:48.994693 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:35:48.994693 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:35:48.994693 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 22:35:48.994693 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:48.994693 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 396/784] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Jul 21 22:35:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:48.995331 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7ed6c2-c4f3-4612-aa29-bcc7c0d289b8 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:49.040272 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:49.047492 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 389/785] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:35:49.053043 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02af6a8-d600-400d-a95b-9a96a5941b12 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:49.101936 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7ed6c2-c4f3-4612-aa29-bcc7c0d289b8 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:49.140047 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05393945-79dc-4059-ad94-dff5a6f1840b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:49.140882 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05393945-79dc-4059-ad94-dff5a6f1840b None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:49.141264 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05393945-79dc-4059-ad94-dff5a6f1840b None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:49.141636 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05393945-79dc-4059-ad94-dff5a6f1840b None admin] PATH_INFO: `/v3/services` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:49.142450 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05393945-79dc-4059-ad94-dff5a6f1840b None admin] RBAC: Adding query filter params (name=network) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:49.142903 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05393945-79dc-4059-ad94-dff5a6f1840b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:49.144357 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05393945-79dc-4059-ad94-dff5a6f1840b None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:49.150217 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 397/786] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Jul 21 22:35:48 2022] GET /identity/v3/services?name=network => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:49.155634 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05393945-79dc-4059-ad94-dff5a6f1840b None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:49.171127 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:49.186274 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02af6a8-d600-400d-a95b-9a96a5941b12 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:49.220708 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67530a78-52fd-4dcd-904c-445d508a2410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:49.221810 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67530a78-52fd-4dcd-904c-445d508a2410 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:49.222330 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67530a78-52fd-4dcd-904c-445d508a2410 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:49.222831 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67530a78-52fd-4dcd-904c-445d508a2410 None admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:49.223890 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67530a78-52fd-4dcd-904c-445d508a2410 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:49.224487 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67530a78-52fd-4dcd-904c-445d508a2410 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:49.226118 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67530a78-52fd-4dcd-904c-445d508a2410 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:49.244957 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 390/787] 10.222.0.52 () {56 vars in 1149 bytes} [Thu Jul 21 22:35:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:49.246117 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67530a78-52fd-4dcd-904c-445d508a2410 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:49.290609 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05393945-79dc-4059-ad94-dff5a6f1840b None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:49.294910 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:49.341861 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b562a23-565d-464b-9512-5798ca35ba6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:49.343291 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b562a23-565d-464b-9512-5798ca35ba6e None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:49.343592 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b562a23-565d-464b-9512-5798ca35ba6e None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:49.344000 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b562a23-565d-464b-9512-5798ca35ba6e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:49.358867 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b562a23-565d-464b-9512-5798ca35ba6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:35:49.369403 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67530a78-52fd-4dcd-904c-445d508a2410 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:49.369654 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-6b562a23-565d-464b-9512-5798ca35ba6e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:35:49.369654 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:49.369654 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:49.369654 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:49.369654 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:49.369654 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:49.369654 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:49.369654 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:49.369654 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:49.369654 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:49.369654 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:49.369654 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:49.369654 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:35:49.369654 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:35:49.369654 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:35:49.369654 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:35:49.370572 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:35:49.370572 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:35:49.370572 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:35:49.370572 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:35:49.370572 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:49.370572 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:49.370572 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:49.370572 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:49.370572 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:49.370572 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:49.370572 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:49.370572 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:49.370572 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:49.370572 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:49.370572 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:49.370572 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:49.371449 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:49.371449 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:49.371449 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:49.371449 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:49.371449 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:35:49.371449 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:35:49.371449 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:35:49.371449 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:35:49.371449 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:35:49.371449 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:35:49.371449 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:35:49.371449 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:49.371449 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 398/788] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 22:35:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:49.372215 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b562a23-565d-464b-9512-5798ca35ba6e None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:49.421980 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e17275b-1f85-48f3-9b6c-04acb884c3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:49.422663 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e17275b-1f85-48f3-9b6c-04acb884c3d3 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:49.422975 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e17275b-1f85-48f3-9b6c-04acb884c3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:49.423253 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e17275b-1f85-48f3-9b6c-04acb884c3d3 None admin] PATH_INFO: `/v3/services` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:49.423966 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e17275b-1f85-48f3-9b6c-04acb884c3d3 None admin] RBAC: Adding query filter params (type=network) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:49.424275 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e17275b-1f85-48f3-9b6c-04acb884c3d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:49.425638 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e17275b-1f85-48f3-9b6c-04acb884c3d3 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:49.437919 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 391/789] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Jul 21 22:35:49 2022] GET /identity/v3/services?type=network => generated 350 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:49.437919 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e17275b-1f85-48f3-9b6c-04acb884c3d3 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:49.458700 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:49.507434 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b562a23-565d-464b-9512-5798ca35ba6e None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:49.545990 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf391f7-7776-4b83-9dad-772245a41a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:49.547243 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf391f7-7776-4b83-9dad-772245a41a68 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:49.547853 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf391f7-7776-4b83-9dad-772245a41a68 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:49.548337 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf391f7-7776-4b83-9dad-772245a41a68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:49.560625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf391f7-7776-4b83-9dad-772245a41a68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:49.562550 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf391f7-7776-4b83-9dad-772245a41a68 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:49.564309 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e17275b-1f85-48f3-9b6c-04acb884c3d3 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:49.582508 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-acf391f7-7776-4b83-9dad-772245a41a68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:49.582508 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:49.582508 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:49.582508 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:49.582508 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:49.582508 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:49.582508 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:49.582508 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:49.582508 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:49.582508 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:49.582508 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:49.582508 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:49.582508 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:49.582508 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:49.582508 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:49.583997 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:49.583997 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:49.583997 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:49.583997 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:49.583997 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:49.583997 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:49.583997 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:49.583997 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:49.583997 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:49.583997 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:49.583997 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:49.583997 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:49.583997 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:49.583997 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:49.583997 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:49.583997 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:49.583997 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:49.584904 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:49.584904 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:49.584904 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:49.584904 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:49.591914 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 399/790] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:49.593191 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf391f7-7776-4b83-9dad-772245a41a68 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:49.602684 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e904db9-1893-4fe5-824a-40cd002e6f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:49.603460 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e904db9-1893-4fe5-824a-40cd002e6f33 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:49.603745 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e904db9-1893-4fe5-824a-40cd002e6f33 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:49.604021 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e904db9-1893-4fe5-824a-40cd002e6f33 None admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:49.604770 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e904db9-1893-4fe5-824a-40cd002e6f33 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:49.605076 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e904db9-1893-4fe5-824a-40cd002e6f33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:49.606987 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e904db9-1893-4fe5-824a-40cd002e6f33 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:49.616073 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:49.621445 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 392/791] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:35:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:49.621445 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e904db9-1893-4fe5-824a-40cd002e6f33 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:49.726851 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf391f7-7776-4b83-9dad-772245a41a68 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:49.739136 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:49.757553 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e904db9-1893-4fe5-824a-40cd002e6f33 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:49.769771 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c91bb0-42b2-4545-99f7-1ffcdf8a4192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:49.770761 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c91bb0-42b2-4545-99f7-1ffcdf8a4192 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:49.771532 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c91bb0-42b2-4545-99f7-1ffcdf8a4192 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:49.771887 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c91bb0-42b2-4545-99f7-1ffcdf8a4192 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:49.772751 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c91bb0-42b2-4545-99f7-1ffcdf8a4192 None admin] RBAC: Adding query filter params (interface=public, service_id=75665ee2341f457dbee48201808e1efe, region_id=RegionOne) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:49.773131 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c91bb0-42b2-4545-99f7-1ffcdf8a4192 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:49.774594 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c91bb0-42b2-4545-99f7-1ffcdf8a4192 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:49.787806 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 400/792] 10.222.0.52 () {56 vars in 1251 bytes} [Thu Jul 21 22:35:49 2022] GET /identity/v3/endpoints?service_id=75665ee2341f457dbee48201808e1efe&interface=public®ion_id=RegionOne => generated 194 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:49.788905 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c91bb0-42b2-4545-99f7-1ffcdf8a4192 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:49.805652 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea46077f-1d50-4f96-be83-f106e9f0a2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:49.806899 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46077f-1d50-4f96-be83-f106e9f0a2ad None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:49.807826 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46077f-1d50-4f96-be83-f106e9f0a2ad None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:49.808452 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46077f-1d50-4f96-be83-f106e9f0a2ad None admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:49.809445 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea46077f-1d50-4f96-be83-f106e9f0a2ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:49.810147 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea46077f-1d50-4f96-be83-f106e9f0a2ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:49.812079 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea46077f-1d50-4f96-be83-f106e9f0a2ad None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:49.830911 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 393/793] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:49.831987 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea46077f-1d50-4f96-be83-f106e9f0a2ad None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:49.884804 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:49.913560 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c91bb0-42b2-4545-99f7-1ffcdf8a4192 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:49.951740 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea46077f-1d50-4f96-be83-f106e9f0a2ad None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:49.956380 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d413bfb-a9d7-44a5-8186-ff2b1aa4ee56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:49.957365 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d413bfb-a9d7-44a5-8186-ff2b1aa4ee56 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:49.957804 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d413bfb-a9d7-44a5-8186-ff2b1aa4ee56 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:49.958230 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d413bfb-a9d7-44a5-8186-ff2b1aa4ee56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:49.971918 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d413bfb-a9d7-44a5-8186-ff2b1aa4ee56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:49.973939 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d413bfb-a9d7-44a5-8186-ff2b1aa4ee56 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:49.988699 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa9134-2bdd-42b8-b455-94073b1de630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:49.989470 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa9134-2bdd-42b8-b455-94073b1de630 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:49.990541 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa9134-2bdd-42b8-b455-94073b1de630 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:49.990980 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa9134-2bdd-42b8-b455-94073b1de630 None admin] PATH_INFO: `/v3/services` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:49.991762 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fa9134-2bdd-42b8-b455-94073b1de630 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:49.993241 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fa9134-2bdd-42b8-b455-94073b1de630 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:49.996592 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-9d413bfb-a9d7-44a5-8186-ff2b1aa4ee56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:49.996592 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:49.996592 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:49.996592 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:49.996592 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:49.996592 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:49.996592 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:49.996592 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:49.996592 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:49.996592 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:49.996592 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:49.996592 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:49.996592 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:49.996592 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:49.996592 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:49.998318 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:49.998318 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:49.998318 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:49.998318 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:49.998318 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:49.998318 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:49.998318 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:49.998318 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:49.998318 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:49.998318 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:49.998318 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:49.998318 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:49.998318 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:49.998318 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:49.998318 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:49.998318 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:49.998318 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:49.999617 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:49.999617 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:49.999617 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:49.999617 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:50.001662 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 401/794] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:50.002658 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d413bfb-a9d7-44a5-8186-ff2b1aa4ee56 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:50.008645 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 394/795] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Jul 21 22:35:49 2022] GET /identity/v3/services => generated 772 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:50.008645 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa9134-2bdd-42b8-b455-94073b1de630 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:50.018077 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:50.134875 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d413bfb-a9d7-44a5-8186-ff2b1aa4ee56 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:50.194984 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:50.206213 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc17770-b989-4336-beb0-528afc3696f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:50.208972 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc17770-b989-4336-beb0-528afc3696f8 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:50.208972 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc17770-b989-4336-beb0-528afc3696f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:50.212544 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc17770-b989-4336-beb0-528afc3696f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:50.227636 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc17770-b989-4336-beb0-528afc3696f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:50.230291 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc17770-b989-4336-beb0-528afc3696f8 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:50.242358 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa9134-2bdd-42b8-b455-94073b1de630 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:50.256155 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-ebc17770-b989-4336-beb0-528afc3696f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:35:50.256155 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:50.256155 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:50.256155 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:50.256155 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:50.256155 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:50.256155 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:50.256155 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:50.256155 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:50.256155 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:50.256155 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:50.256155 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:50.256155 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:35:50.256155 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:35:50.256155 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:35:50.257088 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:35:50.257088 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:50.257088 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:50.257088 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:50.257088 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:50.257088 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:50.257088 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:50.257088 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:50.257088 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:50.257088 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:50.257088 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:50.257088 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:50.257088 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:50.257088 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:50.257088 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:50.257088 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:50.257088 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:50.258003 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:35:50.258003 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:35:50.258003 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:35:50.258003 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:35:50.258003 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:35:50.258003 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:35:50.258003 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:35:50.258003 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:50.262101 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 402/796] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:35:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:50.268394 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc17770-b989-4336-beb0-528afc3696f8 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:50.293710 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd990a-bb25-4e2e-bea2-f8e480bbd389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:50.294388 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd990a-bb25-4e2e-bea2-f8e480bbd389 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:50.294675 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd990a-bb25-4e2e-bea2-f8e480bbd389 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:50.294984 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd990a-bb25-4e2e-bea2-f8e480bbd389 None admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:50.296177 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bd990a-bb25-4e2e-bea2-f8e480bbd389 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:50.296492 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bd990a-bb25-4e2e-bea2-f8e480bbd389 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:50.298144 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bd990a-bb25-4e2e-bea2-f8e480bbd389 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:50.316258 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 395/797] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:50.321513 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd990a-bb25-4e2e-bea2-f8e480bbd389 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:50.390622 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc17770-b989-4336-beb0-528afc3696f8 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:50.397435 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:50.428840 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1540d3-10fc-434e-8e04-81bba79e5ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:50.429666 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1540d3-10fc-434e-8e04-81bba79e5ead None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:50.430054 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1540d3-10fc-434e-8e04-81bba79e5ead None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:50.430432 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1540d3-10fc-434e-8e04-81bba79e5ead None admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:50.431382 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1540d3-10fc-434e-8e04-81bba79e5ead None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:50.431795 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1540d3-10fc-434e-8e04-81bba79e5ead None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:50.433331 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1540d3-10fc-434e-8e04-81bba79e5ead None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:50.447478 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 403/798] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 22:35:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:35:50.453784 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1540d3-10fc-434e-8e04-81bba79e5ead None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:50.475465 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd990a-bb25-4e2e-bea2-f8e480bbd389 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:50.546351 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d88c532-be3f-4ac9-a28c-886cbc84089c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:50.547211 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88c532-be3f-4ac9-a28c-886cbc84089c None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:50.547499 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88c532-be3f-4ac9-a28c-886cbc84089c None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:50.547777 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88c532-be3f-4ac9-a28c-886cbc84089c None admin] PATH_INFO: `/v3/users/swift` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:50.554356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:50.558286 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d88c532-be3f-4ac9-a28c-886cbc84089c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:50.559791 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d88c532-be3f-4ac9-a28c-886cbc84089c None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:50.577831 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-8d88c532-be3f-4ac9-a28c-886cbc84089c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 22:35:50.577831 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:50.577831 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:50.577831 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:50.577831 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:50.577831 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:50.577831 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:50.577831 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:50.577831 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:50.577831 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:50.577831 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:50.577831 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:50.577831 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:35:50.577831 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:35:50.577831 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:35:50.577831 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:35:50.579326 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:50.579326 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:50.579326 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:35:50.579326 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:35:50.579326 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:35:50.579326 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:35:50.579326 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:50.579326 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:50.579326 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:50.579326 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:50.579326 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:50.579326 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:50.579326 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:50.579326 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:50.579326 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:50.579326 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:50.579326 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:50.579326 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:50.580255 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:50.580255 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:50.580255 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:35:50.580255 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:35:50.580255 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:35:50.580255 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:35:50.580255 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:35:50.580255 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:35:50.580255 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 22:35:50.580255 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:50.580255 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 396/799] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 22:35:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:35:50.586430 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d88c532-be3f-4ac9-a28c-886cbc84089c None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:50.661132 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1540d3-10fc-434e-8e04-81bba79e5ead None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:50.727012 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b52573c-91dd-42de-af80-5557364094f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:50.728150 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b52573c-91dd-42de-af80-5557364094f9 None admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:50.728577 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b52573c-91dd-42de-af80-5557364094f9 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:50.728996 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b52573c-91dd-42de-af80-5557364094f9 None admin] PATH_INFO: `/v3/projects/aa11ab8229b642ce962193b74001d3bc/users/63cc904578554aea84ff1917a5ed075e/roles/715d866890d14923b11470d544b4f960` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:50.753784 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b52573c-91dd-42de-af80-5557364094f9 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa11ab8229b642ce962193b74001d3bc, user_id=63cc904578554aea84ff1917a5ed075e, role_id=715d866890d14923b11470d544b4f960)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:50.756090 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b52573c-91dd-42de-af80-5557364094f9 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:50.758245 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d88c532-be3f-4ac9-a28c-886cbc84089c None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:50.784124 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:50.794961 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce011ed2-99d6-42eb-b361-545cfba537fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:50.795637 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce011ed2-99d6-42eb-b361-545cfba537fe None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:50.795921 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce011ed2-99d6-42eb-b361-545cfba537fe None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:50.796203 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce011ed2-99d6-42eb-b361-545cfba537fe None admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:50.797039 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce011ed2-99d6-42eb-b361-545cfba537fe None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:50.797352 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce011ed2-99d6-42eb-b361-545cfba537fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:50.798801 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce011ed2-99d6-42eb-b361-545cfba537fe None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:50.816870 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 397/800] 10.222.0.52 () {56 vars in 1135 bytes} [Thu Jul 21 22:35:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:50.818370 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c363b45-16ce-4b36-814f-ddf143514edf None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:50.818659 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c363b45-16ce-4b36-814f-ddf143514edf None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:50.818970 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c363b45-16ce-4b36-814f-ddf143514edf None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:50.820590 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 398/801] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:50.821481 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50072ee3-f35e-4db8-848b-7af14074009c None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:50.821767 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50072ee3-f35e-4db8-848b-7af14074009c None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:50.822041 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50072ee3-f35e-4db8-848b-7af14074009c None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:50.825429 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 399/802] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:50.825429 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50072ee3-f35e-4db8-848b-7af14074009c None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:50.839855 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 404/803] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 21 22:35:50 2022] PUT /identity/v3/projects/aa11ab8229b642ce962193b74001d3bc/users/63cc904578554aea84ff1917a5ed075e/roles/715d866890d14923b11470d544b4f960 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:35:50.841313 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8b48c-7493-4c54-ad4b-0daafd7af07f None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:50.841612 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8b48c-7493-4c54-ad4b-0daafd7af07f None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:50.841891 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8b48c-7493-4c54-ad4b-0daafd7af07f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:50.862050 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-80a8b48c-7493-4c54-ad4b-0daafd7af07f None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:50.863513 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-80a8b48c-7493-4c54-ad4b-0daafd7af07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:50.946884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 405/804] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:35:50.946884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc8f15-7742-494f-9ef8-1a0f0f5bb458 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:50.946884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc8f15-7742-494f-9ef8-1a0f0f5bb458 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:50.946884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc8f15-7742-494f-9ef8-1a0f0f5bb458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:50.977042 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-22cc8f15-7742-494f-9ef8-1a0f0f5bb458 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:50.978270 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-22cc8f15-7742-494f-9ef8-1a0f0f5bb458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:50.993616 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50072ee3-f35e-4db8-848b-7af14074009c None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:51.010951 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:51.033977 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0595f9db-7693-44ad-8cca-23469109fe7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:51.034817 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595f9db-7693-44ad-8cca-23469109fe7d None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:51.035091 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595f9db-7693-44ad-8cca-23469109fe7d None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:51.035443 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595f9db-7693-44ad-8cca-23469109fe7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:51.046553 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0595f9db-7693-44ad-8cca-23469109fe7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:51.048054 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0595f9db-7693-44ad-8cca-23469109fe7d None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:51.070794 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 406/805] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:35:51.070794 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cc8f15-7742-494f-9ef8-1a0f0f5bb458 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:51.077373 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-0595f9db-7693-44ad-8cca-23469109fe7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:51.077373 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:51.077373 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:51.077373 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:51.077373 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:51.077373 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:51.077373 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:51.077373 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:51.077373 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:51.077373 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:51.077373 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:51.077373 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:51.077373 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:51.077373 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:51.077373 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:51.078239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:51.078239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:51.078239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:51.078239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:51.078239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:51.078239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:51.078239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:51.078239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:51.078239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:51.078239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:51.078239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:51.078239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:51.078239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:51.078239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:51.078239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:51.078239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:51.078239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:51.079599 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:51.079599 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:51.079599 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:51.079599 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:51.079599 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 400/806] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:51.085247 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0595f9db-7693-44ad-8cca-23469109fe7d None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:51.121348 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:51.177028 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cc8f15-7742-494f-9ef8-1a0f0f5bb458 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:51.189515 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0595f9db-7693-44ad-8cca-23469109fe7d None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:51.214515 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f165784b-abcc-4cab-b8d1-376de4a05f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:51.215280 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f165784b-abcc-4cab-b8d1-376de4a05f87 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:51.215566 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f165784b-abcc-4cab-b8d1-376de4a05f87 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:51.215843 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f165784b-abcc-4cab-b8d1-376de4a05f87 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:51.216499 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f165784b-abcc-4cab-b8d1-376de4a05f87 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:51.217872 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f165784b-abcc-4cab-b8d1-376de4a05f87 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:51.224976 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621eb174-ca63-4e09-ad79-c106a47b0d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:51.225635 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621eb174-ca63-4e09-ad79-c106a47b0d70 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:51.225916 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621eb174-ca63-4e09-ad79-c106a47b0d70 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:51.226186 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621eb174-ca63-4e09-ad79-c106a47b0d70 None admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:51.227292 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621eb174-ca63-4e09-ad79-c106a47b0d70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:51.227602 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621eb174-ca63-4e09-ad79-c106a47b0d70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:51.228947 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621eb174-ca63-4e09-ad79-c106a47b0d70 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:51.231355 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-f165784b-abcc-4cab-b8d1-376de4a05f87 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 22:35:51.231355 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:51.231355 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:35:51.231355 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:35:51.231355 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:35:51.231355 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:35:51.231355 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:35:51.231355 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:35:51.231355 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 22:35:51.231355 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:51.231355 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:35:51.231355 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:51.231355 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:51.231355 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:51.231355 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:51.231355 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:51.233518 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:51.233518 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:51.233518 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:51.233518 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:51.233518 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:51.233518 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:51.233518 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:51.233518 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:35:51.233518 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:35:51.233518 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:35:51.233518 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:35:51.233518 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:51.233518 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:51.233518 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:51.233518 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:51.233518 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:51.235529 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:51.235529 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:51.235529 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:51.235529 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:51.235529 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:51.235529 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:51.235529 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:51.235529 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:51.235529 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:51.235529 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:51.235529 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:51.235529 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:35:51.235529 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:35:51.235529 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 22:35:51.235529 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:51.235529 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 407/807] 10.222.0.52 () {56 vars in 1129 bytes} [Thu Jul 21 22:35:51 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:51.237735 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f165784b-abcc-4cab-b8d1-376de4a05f87 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:51.251507 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 401/808] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:35:51.252298 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621eb174-ca63-4e09-ad79-c106a47b0d70 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:51.263974 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:51.345175 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f165784b-abcc-4cab-b8d1-376de4a05f87 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:51.357297 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:51.364475 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621eb174-ca63-4e09-ad79-c106a47b0d70 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:51.384262 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc28f3d-13c1-431e-b3b7-c0e689b0af5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:51.385305 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc28f3d-13c1-431e-b3b7-c0e689b0af5e None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:51.385768 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc28f3d-13c1-431e-b3b7-c0e689b0af5e None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:51.386244 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc28f3d-13c1-431e-b3b7-c0e689b0af5e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:51.398373 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc28f3d-13c1-431e-b3b7-c0e689b0af5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:35:51.408408 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b2422-5280-4cdc-97fb-e02e4e3630eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:51.409231 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b2422-5280-4cdc-97fb-e02e4e3630eb None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:51.409674 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b2422-5280-4cdc-97fb-e02e4e3630eb None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:51.410055 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b2422-5280-4cdc-97fb-e02e4e3630eb None admin] PATH_INFO: `/v3/services` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:51.410965 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b2422-5280-4cdc-97fb-e02e4e3630eb None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:51.411380 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b2422-5280-4cdc-97fb-e02e4e3630eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:51.412941 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b2422-5280-4cdc-97fb-e02e4e3630eb None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:51.413343 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-3cc28f3d-13c1-431e-b3b7-c0e689b0af5e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:35:51.413343 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:51.413343 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:51.413343 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:51.413343 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:51.413343 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:51.413343 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:51.413343 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:51.413343 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:51.413343 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:51.413343 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:51.413343 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:51.413343 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:35:51.413343 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:35:51.413343 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:35:51.413343 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:35:51.414424 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:35:51.414424 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:35:51.414424 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:35:51.414424 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:35:51.414424 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:51.414424 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:51.414424 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:51.414424 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:51.414424 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:51.414424 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:51.414424 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:51.414424 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:51.414424 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:51.414424 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:51.414424 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:51.414424 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:51.415459 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:51.415459 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:51.415459 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:51.415459 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:51.415459 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:35:51.415459 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:35:51.415459 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:35:51.415459 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:35:51.415459 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:35:51.415459 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:35:51.415459 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:35:51.415459 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:51.417902 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 408/809] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 22:35:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:51.418964 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc28f3d-13c1-431e-b3b7-c0e689b0af5e None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:51.424739 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 402/810] 10.222.0.52 () {56 vars in 1124 bytes} [Thu Jul 21 22:35:51 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:51.425924 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b2422-5280-4cdc-97fb-e02e4e3630eb None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:51.525454 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:51.536987 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b2422-5280-4cdc-97fb-e02e4e3630eb None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:51.541668 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc28f3d-13c1-431e-b3b7-c0e689b0af5e None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:51.573584 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae7e6d3-75b7-4d0d-8ac1-303cd25c3fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:51.574503 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7e6d3-75b7-4d0d-8ac1-303cd25c3fa8 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:51.575007 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7e6d3-75b7-4d0d-8ac1-303cd25c3fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:51.575500 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7e6d3-75b7-4d0d-8ac1-303cd25c3fa8 None admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:51.576620 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae7e6d3-75b7-4d0d-8ac1-303cd25c3fa8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:51.577202 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae7e6d3-75b7-4d0d-8ac1-303cd25c3fa8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:51.579029 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae7e6d3-75b7-4d0d-8ac1-303cd25c3fa8 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:51.579562 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07461a30-23d8-4fda-ac0a-aee3345616f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:51.579905 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07461a30-23d8-4fda-ac0a-aee3345616f4 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:51.580193 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07461a30-23d8-4fda-ac0a-aee3345616f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:51.580545 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07461a30-23d8-4fda-ac0a-aee3345616f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:51.588856 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07461a30-23d8-4fda-ac0a-aee3345616f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:51.590487 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07461a30-23d8-4fda-ac0a-aee3345616f4 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:51.591356 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 403/811] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:35:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:51.592495 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae7e6d3-75b7-4d0d-8ac1-303cd25c3fa8 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:51.610689 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-07461a30-23d8-4fda-ac0a-aee3345616f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:35:51.610689 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:51.610689 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:51.610689 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:51.610689 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:51.610689 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:51.610689 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:51.610689 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:51.610689 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:51.610689 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:51.610689 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:51.610689 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:51.610689 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:35:51.610689 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:35:51.610689 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:35:51.611629 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:35:51.611629 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:51.611629 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:51.611629 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:51.611629 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:51.611629 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:51.611629 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:51.611629 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:51.611629 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:51.611629 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:51.611629 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:51.611629 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:51.611629 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:51.611629 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:51.611629 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:51.611629 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:51.611629 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:51.613044 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:35:51.613044 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:35:51.613044 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:35:51.613044 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:35:51.613044 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:35:51.613044 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:35:51.613044 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:35:51.613044 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:51.613044 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 409/812] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:35:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:51.613534 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07461a30-23d8-4fda-ac0a-aee3345616f4 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:51.619922 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:51.706394 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae7e6d3-75b7-4d0d-8ac1-303cd25c3fa8 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:51.722417 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07461a30-23d8-4fda-ac0a-aee3345616f4 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:51.742979 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fe06d6-bdc7-4ff3-bd90-2691ffd90552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:51.743914 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fe06d6-bdc7-4ff3-bd90-2691ffd90552 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:51.744407 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fe06d6-bdc7-4ff3-bd90-2691ffd90552 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:51.744889 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fe06d6-bdc7-4ff3-bd90-2691ffd90552 None admin] PATH_INFO: `/v3/services` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:51.745851 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fe06d6-bdc7-4ff3-bd90-2691ffd90552 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:51.746389 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fe06d6-bdc7-4ff3-bd90-2691ffd90552 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:51.747991 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fe06d6-bdc7-4ff3-bd90-2691ffd90552 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:51.753740 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 404/813] 10.222.0.52 () {56 vars in 1124 bytes} [Thu Jul 21 22:35:51 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:51.754891 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fe06d6-bdc7-4ff3-bd90-2691ffd90552 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:51.760082 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63f04ae-a372-4207-9815-6fc93dd35abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:51.760252 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63f04ae-a372-4207-9815-6fc93dd35abb None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:51.760433 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63f04ae-a372-4207-9815-6fc93dd35abb None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:51.760712 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63f04ae-a372-4207-9815-6fc93dd35abb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:51.772440 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63f04ae-a372-4207-9815-6fc93dd35abb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:51.773889 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63f04ae-a372-4207-9815-6fc93dd35abb None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:51.780543 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:51.792826 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-d63f04ae-a372-4207-9815-6fc93dd35abb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:51.792826 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:51.792826 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:51.792826 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:51.792826 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:51.792826 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:51.792826 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:51.792826 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:51.792826 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:51.792826 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:51.792826 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:51.792826 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:51.792826 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:51.792826 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:51.792826 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:51.799564 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:51.799564 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:51.799564 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:51.799564 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:51.799564 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:51.799564 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:51.799564 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:51.799564 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:51.799564 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:51.799564 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:51.799564 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:51.799564 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:51.799564 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:51.799564 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:51.799564 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:51.799564 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:51.799564 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:51.800479 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:51.800479 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:51.800479 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:51.800479 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:51.801341 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 410/814] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:51.802964 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63f04ae-a372-4207-9815-6fc93dd35abb None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:51.850238 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:51.869075 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fe06d6-bdc7-4ff3-bd90-2691ffd90552 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:51.906827 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0cacbe-56c7-4bc5-8bf9-f842549da653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:51.907507 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0cacbe-56c7-4bc5-8bf9-f842549da653 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:51.907794 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0cacbe-56c7-4bc5-8bf9-f842549da653 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:51.908073 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0cacbe-56c7-4bc5-8bf9-f842549da653 None admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:51.908863 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0cacbe-56c7-4bc5-8bf9-f842549da653 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:51.909175 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0cacbe-56c7-4bc5-8bf9-f842549da653 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:51.910619 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0cacbe-56c7-4bc5-8bf9-f842549da653 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:51.911349 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63f04ae-a372-4207-9815-6fc93dd35abb None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:51.925110 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 405/815] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 22:35:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:51.926005 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0cacbe-56c7-4bc5-8bf9-f842549da653 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:51.948774 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52abc277-f6d7-49e0-8d24-df40732fdc03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:51.949779 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52abc277-f6d7-49e0-8d24-df40732fdc03 None admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:51.950290 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52abc277-f6d7-49e0-8d24-df40732fdc03 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:51.950820 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52abc277-f6d7-49e0-8d24-df40732fdc03 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:51.951945 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52abc277-f6d7-49e0-8d24-df40732fdc03 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:51.953536 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52abc277-f6d7-49e0-8d24-df40732fdc03 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:52.033750 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0cacbe-56c7-4bc5-8bf9-f842549da653 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:52.072678 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc11fa5-f0b5-4512-801a-a70d4de40e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:52.073373 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc11fa5-f0b5-4512-801a-a70d4de40e43 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:52.073664 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc11fa5-f0b5-4512-801a-a70d4de40e43 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:52.073934 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc11fa5-f0b5-4512-801a-a70d4de40e43 None admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:52.074676 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc11fa5-f0b5-4512-801a-a70d4de40e43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:52.075030 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc11fa5-f0b5-4512-801a-a70d4de40e43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:52.082204 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc11fa5-f0b5-4512-801a-a70d4de40e43 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:52.082395 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 411/816] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Jul 21 22:35:51 2022] POST /identity/v3/endpoints => generated 346 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:52.082395 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52abc277-f6d7-49e0-8d24-df40732fdc03 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:52.104346 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 406/817] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:52.110373 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc11fa5-f0b5-4512-801a-a70d4de40e43 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:52.116597 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:52.192274 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:52.248581 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52abc277-f6d7-49e0-8d24-df40732fdc03 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:52.286654 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43342709-bce9-4678-8074-4379cd37d6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:52.287165 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43342709-bce9-4678-8074-4379cd37d6bd None admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:52.288193 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43342709-bce9-4678-8074-4379cd37d6bd None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:52.288473 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43342709-bce9-4678-8074-4379cd37d6bd None admin] PATH_INFO: `/v3/projects/aa11ab8229b642ce962193b74001d3bc/users/ba7eef4e1b27476ab4016449d9c3a4b9/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:52.290454 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43342709-bce9-4678-8074-4379cd37d6bd None admin] RBAC: Authorizing `identity:create_grant(project_id=aa11ab8229b642ce962193b74001d3bc, user_id=ba7eef4e1b27476ab4016449d9c3a4b9, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:52.291928 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43342709-bce9-4678-8074-4379cd37d6bd None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:52.315988 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc11fa5-f0b5-4512-801a-a70d4de40e43 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:52.354845 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6436377b-8918-4a1b-92fd-9a5333931bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:52.356037 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436377b-8918-4a1b-92fd-9a5333931bce None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:52.356582 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436377b-8918-4a1b-92fd-9a5333931bce None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:52.357062 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436377b-8918-4a1b-92fd-9a5333931bce None admin] PATH_INFO: `/v3/users/nova` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:52.365966 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6436377b-8918-4a1b-92fd-9a5333931bce None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:52.367753 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6436377b-8918-4a1b-92fd-9a5333931bce None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:52.377276 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 412/818] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 21 22:35:52 2022] PUT /identity/v3/projects/aa11ab8229b642ce962193b74001d3bc/users/ba7eef4e1b27476ab4016449d9c3a4b9/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:35:52.383679 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-6436377b-8918-4a1b-92fd-9a5333931bce None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:35:52.383679 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:52.383679 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:52.383679 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:52.383679 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:52.383679 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:52.383679 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:52.383679 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:52.383679 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:52.383679 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:52.383679 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:52.383679 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:52.383679 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:35:52.383679 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:35:52.383679 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:35:52.383679 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:35:52.384687 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:52.384687 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:52.384687 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:35:52.384687 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:35:52.384687 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:35:52.384687 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:35:52.384687 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:52.384687 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:52.384687 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:52.384687 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:52.384687 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:52.384687 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:52.384687 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:52.384687 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:52.384687 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:52.384687 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:52.384687 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:52.384687 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:52.385727 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:52.385727 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:52.385727 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:35:52.385727 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:35:52.385727 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:35:52.385727 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:35:52.385727 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:35:52.385727 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:35:52.385727 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:35:52.385727 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:52.388622 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 407/819] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Jul 21 22:35:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:35:52.394703 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43342709-bce9-4678-8074-4379cd37d6bd None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:52.420191 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb1ce96-ec24-442f-8da1-17c2f42a408c None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:52.420858 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb1ce96-ec24-442f-8da1-17c2f42a408c None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:52.421342 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb1ce96-ec24-442f-8da1-17c2f42a408c None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:52.423488 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 408/820] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:35:52.429224 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4fb544-aec4-4432-a1e5-fc23fc9a13f6 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:52.429847 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4fb544-aec4-4432-a1e5-fc23fc9a13f6 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:52.430389 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4fb544-aec4-4432-a1e5-fc23fc9a13f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:52.457700 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-4d4fb544-aec4-4432-a1e5-fc23fc9a13f6 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:52.459594 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-4d4fb544-aec4-4432-a1e5-fc23fc9a13f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:52.579478 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43342709-bce9-4678-8074-4379cd37d6bd None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:52.601087 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:52.606050 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:52.616321 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 409/821] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:35:52.632270 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb10f00-cff8-4512-9f87-125694704989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:52.633333 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb10f00-cff8-4512-9f87-125694704989 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:52.633889 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb10f00-cff8-4512-9f87-125694704989 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:52.634435 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb10f00-cff8-4512-9f87-125694704989 None admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:52.635527 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb10f00-cff8-4512-9f87-125694704989 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:52.636044 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb10f00-cff8-4512-9f87-125694704989 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:52.637797 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb10f00-cff8-4512-9f87-125694704989 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:52.639825 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4fb544-aec4-4432-a1e5-fc23fc9a13f6 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:52.671723 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 413/822] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Jul 21 22:35:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:52.678892 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb10f00-cff8-4512-9f87-125694704989 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:52.763547 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4fb544-aec4-4432-a1e5-fc23fc9a13f6 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:52.798331 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb10f00-cff8-4512-9f87-125694704989 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:52.809435 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0430b3-6c28-421f-8e7f-9a0399125661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:52.810305 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0430b3-6c28-421f-8e7f-9a0399125661 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:52.810582 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0430b3-6c28-421f-8e7f-9a0399125661 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:52.810880 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0430b3-6c28-421f-8e7f-9a0399125661 None admin] PATH_INFO: `/v3/services/network` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:52.811542 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0430b3-6c28-421f-8e7f-9a0399125661 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:52.812909 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0430b3-6c28-421f-8e7f-9a0399125661 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:52.836621 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-ad0430b3-6c28-421f-8e7f-9a0399125661 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 22:35:52.836621 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:52.836621 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:35:52.836621 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:35:52.836621 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:35:52.836621 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:35:52.836621 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:35:52.836621 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:35:52.836621 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 22:35:52.836621 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:52.836621 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:35:52.836621 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:52.836621 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:52.836621 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:52.836621 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:52.836621 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:52.836621 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:52.837657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:52.837657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:52.837657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:52.837657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:52.837657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:52.837657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:52.837657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:35:52.837657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:35:52.837657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:35:52.837657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:35:52.837657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:52.837657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:52.837657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:52.837657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:52.837657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:52.837657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:52.838482 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:52.838482 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:52.838482 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:52.838482 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:52.838482 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:52.838482 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:52.838482 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:52.838482 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:52.838482 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:52.838482 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:52.838482 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:35:52.838482 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:35:52.838482 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 22:35:52.838482 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:52.838482 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 410/823] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:35:52 2022] GET /identity/v3/services/network => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:35:52.842040 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201edda5-e35b-41a5-904c-49370e5a8f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:52.843042 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201edda5-e35b-41a5-904c-49370e5a8f47 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:52.843531 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201edda5-e35b-41a5-904c-49370e5a8f47 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:52.843653 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0430b3-6c28-421f-8e7f-9a0399125661 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:52.844120 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201edda5-e35b-41a5-904c-49370e5a8f47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:52.882862 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:52.921083 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201edda5-e35b-41a5-904c-49370e5a8f47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:52.923061 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201edda5-e35b-41a5-904c-49370e5a8f47 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:52.971554 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:53.038598 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 414/824] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:35:52 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 22:35:53.049044 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201edda5-e35b-41a5-904c-49370e5a8f47 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:53.081770 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0430b3-6c28-421f-8e7f-9a0399125661 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:53.128150 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:53.131844 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f661aa17-db83-45a0-b4b6-7ca1dccfd91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:53.133533 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661aa17-db83-45a0-b4b6-7ca1dccfd91b None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:53.134392 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661aa17-db83-45a0-b4b6-7ca1dccfd91b None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:53.134392 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661aa17-db83-45a0-b4b6-7ca1dccfd91b None admin] PATH_INFO: `/v3/services` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:53.135483 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f661aa17-db83-45a0-b4b6-7ca1dccfd91b None admin] RBAC: Adding query filter params (name=network) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:53.135998 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f661aa17-db83-45a0-b4b6-7ca1dccfd91b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:53.138009 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f661aa17-db83-45a0-b4b6-7ca1dccfd91b None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:53.145125 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 411/825] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Jul 21 22:35:52 2022] GET /identity/v3/services?name=network => generated 124 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:35:53.147519 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f55eb-5056-4dfd-9b97-eb43632ce36d None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:53.147995 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f55eb-5056-4dfd-9b97-eb43632ce36d None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:53.148719 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f55eb-5056-4dfd-9b97-eb43632ce36d None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:53.150477 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 412/826] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:53.151459 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f55eb-5056-4dfd-9b97-eb43632ce36d None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:53.169172 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201edda5-e35b-41a5-904c-49370e5a8f47 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:53.210900 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81841f41-6e67-4c63-86b0-f67c464a1a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:53.212157 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81841f41-6e67-4c63-86b0-f67c464a1a08 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:53.212629 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81841f41-6e67-4c63-86b0-f67c464a1a08 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:53.213115 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81841f41-6e67-4c63-86b0-f67c464a1a08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:53.224488 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81841f41-6e67-4c63-86b0-f67c464a1a08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:53.226266 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81841f41-6e67-4c63-86b0-f67c464a1a08 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:53.245360 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-81841f41-6e67-4c63-86b0-f67c464a1a08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:53.245360 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:53.245360 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:53.245360 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:53.245360 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:53.245360 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:53.245360 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:53.245360 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:53.245360 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:53.245360 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:53.245360 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:53.245360 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:53.245360 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:53.245360 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:53.245360 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:53.246296 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:53.246296 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:53.246296 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:53.246296 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:53.246296 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:53.246296 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:53.246296 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:53.246296 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:53.246296 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:53.246296 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:53.246296 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:53.246296 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:53.246296 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:53.246296 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:53.246296 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:53.246296 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:53.246296 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:53.247201 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:53.247201 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:53.247201 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:53.247201 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:53.253806 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 415/827] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:35:53.256262 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0afac2-1748-4007-ab15-7bbdf850654e None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:53.256713 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0afac2-1748-4007-ab15-7bbdf850654e None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:53.257139 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0afac2-1748-4007-ab15-7bbdf850654e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:53.280542 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-7d0afac2-1748-4007-ab15-7bbdf850654e None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:53.281833 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-7d0afac2-1748-4007-ab15-7bbdf850654e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:53.361529 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 416/828] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:35:53.362393 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0afac2-1748-4007-ab15-7bbdf850654e None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:53.405669 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f55eb-5056-4dfd-9b97-eb43632ce36d None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:53.432871 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:53.435625 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:53.449902 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e697fa4d-22ac-46b1-92e2-b4663fe93f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:53.450729 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e697fa4d-22ac-46b1-92e2-b4663fe93f52 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:53.451135 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e697fa4d-22ac-46b1-92e2-b4663fe93f52 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:53.451530 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e697fa4d-22ac-46b1-92e2-b4663fe93f52 None admin] PATH_INFO: `/v3/services` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:53.452506 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e697fa4d-22ac-46b1-92e2-b4663fe93f52 None admin] RBAC: Adding query filter params (type=network) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:53.453059 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e697fa4d-22ac-46b1-92e2-b4663fe93f52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:53.455222 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e697fa4d-22ac-46b1-92e2-b4663fe93f52 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:53.462791 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 413/829] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Jul 21 22:35:53 2022] GET /identity/v3/services?type=network => generated 350 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:53.466963 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e697fa4d-22ac-46b1-92e2-b4663fe93f52 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:53.509926 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0afac2-1748-4007-ab15-7bbdf850654e None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:53.547005 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61801ec9-00ab-497f-8549-e43bc7380ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:53.547706 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61801ec9-00ab-497f-8549-e43bc7380ed8 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:53.548038 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61801ec9-00ab-497f-8549-e43bc7380ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:53.548317 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61801ec9-00ab-497f-8549-e43bc7380ed8 None admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:53.549043 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61801ec9-00ab-497f-8549-e43bc7380ed8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:53.549351 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61801ec9-00ab-497f-8549-e43bc7380ed8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:53.550716 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61801ec9-00ab-497f-8549-e43bc7380ed8 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:53.563146 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 417/830] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:53.563925 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61801ec9-00ab-497f-8549-e43bc7380ed8 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:53.578097 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e697fa4d-22ac-46b1-92e2-b4663fe93f52 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:53.614565 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c1c6e-b21d-4b04-aa1f-589cbb0385a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:53.616186 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c1c6e-b21d-4b04-aa1f-589cbb0385a6 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:53.616673 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c1c6e-b21d-4b04-aa1f-589cbb0385a6 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:53.617171 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c1c6e-b21d-4b04-aa1f-589cbb0385a6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:53.628242 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7c1c6e-b21d-4b04-aa1f-589cbb0385a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:35:53.636819 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-5f7c1c6e-b21d-4b04-aa1f-589cbb0385a6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:35:53.636819 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:53.636819 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:53.636819 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:53.636819 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:53.636819 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:53.636819 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:53.636819 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:53.636819 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:53.636819 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:53.636819 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:53.636819 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:53.636819 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:35:53.636819 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:35:53.636819 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:35:53.636819 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:35:53.637833 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:35:53.637833 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:35:53.637833 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:35:53.637833 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:35:53.637833 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:53.637833 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:53.637833 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:53.637833 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:53.637833 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:53.637833 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:53.637833 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:53.637833 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:53.637833 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:53.637833 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:53.637833 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:53.637833 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:53.638657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:53.638657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:53.638657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:53.638657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:53.638657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:35:53.638657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:35:53.638657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:35:53.638657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:35:53.638657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:35:53.638657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:35:53.638657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:35:53.638657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:53.641093 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 414/831] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 21 22:35:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:53.642079 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c1c6e-b21d-4b04-aa1f-589cbb0385a6 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:53.676592 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61801ec9-00ab-497f-8549-e43bc7380ed8 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:53.688464 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:53.692173 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:53.723176 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c078cf3-f8f2-406a-abbe-27e6716e7d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:53.724115 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c078cf3-f8f2-406a-abbe-27e6716e7d56 None admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:53.724584 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c078cf3-f8f2-406a-abbe-27e6716e7d56 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:53.725008 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c078cf3-f8f2-406a-abbe-27e6716e7d56 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:53.726073 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c078cf3-f8f2-406a-abbe-27e6716e7d56 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:53.727888 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c078cf3-f8f2-406a-abbe-27e6716e7d56 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:53.756976 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c1c6e-b21d-4b04-aa1f-589cbb0385a6 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:53.793396 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d9272f-50f6-41ce-9fa4-9b0018ee8d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:53.794216 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d9272f-50f6-41ce-9fa4-9b0018ee8d4e None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:53.794500 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d9272f-50f6-41ce-9fa4-9b0018ee8d4e None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:53.794790 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d9272f-50f6-41ce-9fa4-9b0018ee8d4e None admin] PATH_INFO: `/v3/projects/service` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:53.801802 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d9272f-50f6-41ce-9fa4-9b0018ee8d4e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:53.803305 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d9272f-50f6-41ce-9fa4-9b0018ee8d4e None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:53.809415 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 418/832] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Jul 21 22:35:53 2022] POST /identity/v3/endpoints => generated 327 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:35:53.815360 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c078cf3-f8f2-406a-abbe-27e6716e7d56 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:53.825343 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-28d9272f-50f6-41ce-9fa4-9b0018ee8d4e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:35:53.825343 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:53.825343 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:53.825343 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:53.825343 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:53.825343 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:53.825343 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:53.825343 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:53.825343 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:53.825343 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:53.825343 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:53.825343 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:53.825343 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:35:53.825343 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:35:53.825343 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:35:53.826278 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:35:53.826278 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:53.826278 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:53.826278 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:53.826278 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:53.826278 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:53.826278 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:53.826278 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:53.826278 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:53.826278 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:53.826278 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:53.826278 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:53.826278 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:53.826278 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:53.826278 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:53.826278 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:53.826278 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:53.827620 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:35:53.827620 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:35:53.827620 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:35:53.827620 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:35:53.827620 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:35:53.827620 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:35:53.827620 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:35:53.827620 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:53.835281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 415/833] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:35:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:53.842054 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d9272f-50f6-41ce-9fa4-9b0018ee8d4e None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:53.962832 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c078cf3-f8f2-406a-abbe-27e6716e7d56 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:54.007002 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1661f9-041d-4a1a-a0f0-92ab48dc1210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:54.008282 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1661f9-041d-4a1a-a0f0-92ab48dc1210 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:54.008808 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1661f9-041d-4a1a-a0f0-92ab48dc1210 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:54.009493 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1661f9-041d-4a1a-a0f0-92ab48dc1210 None admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:54.010498 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1661f9-041d-4a1a-a0f0-92ab48dc1210 None admin] RBAC: Adding query filter params (name=service) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:54.011062 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1661f9-041d-4a1a-a0f0-92ab48dc1210 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:54.013583 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1661f9-041d-4a1a-a0f0-92ab48dc1210 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:54.024620 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 419/834] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 21 22:35:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:54.032119 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1661f9-041d-4a1a-a0f0-92ab48dc1210 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:54.065976 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:54.087083 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d9272f-50f6-41ce-9fa4-9b0018ee8d4e None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:54.095562 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:54.133232 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeeae3a-c7a0-49fe-91f6-2263836c2c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:54.134679 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeeae3a-c7a0-49fe-91f6-2263836c2c51 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:54.134996 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeeae3a-c7a0-49fe-91f6-2263836c2c51 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:54.135282 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeeae3a-c7a0-49fe-91f6-2263836c2c51 None admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:54.136101 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeeae3a-c7a0-49fe-91f6-2263836c2c51 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:54.136411 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeeae3a-c7a0-49fe-91f6-2263836c2c51 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:54.138066 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeeae3a-c7a0-49fe-91f6-2263836c2c51 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:54.154353 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 416/835] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 22:35:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:35:54.160097 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeeae3a-c7a0-49fe-91f6-2263836c2c51 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:54.172562 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1661f9-041d-4a1a-a0f0-92ab48dc1210 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:54.215047 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21e53c9-b252-4ba6-abb7-0c8772e2bdb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:54.216272 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21e53c9-b252-4ba6-abb7-0c8772e2bdb6 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:54.216741 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21e53c9-b252-4ba6-abb7-0c8772e2bdb6 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:54.217184 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21e53c9-b252-4ba6-abb7-0c8772e2bdb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:54.230486 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21e53c9-b252-4ba6-abb7-0c8772e2bdb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:54.234134 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21e53c9-b252-4ba6-abb7-0c8772e2bdb6 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:54.254484 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-b21e53c9-b252-4ba6-abb7-0c8772e2bdb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:54.254484 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:54.254484 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:54.254484 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:54.254484 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:54.254484 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:54.254484 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:54.254484 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:54.254484 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:54.254484 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:54.254484 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:54.254484 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:54.254484 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:54.254484 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:54.254484 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:54.256071 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:54.256071 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:54.256071 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:54.256071 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:54.256071 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:54.256071 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:54.256071 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:54.256071 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:54.256071 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:54.256071 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:54.256071 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:54.256071 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:54.256071 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:54.256071 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:54.256071 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:54.256071 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:54.256071 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:54.256934 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:54.256934 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:54.256934 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:54.256934 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:54.261897 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 420/836] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:54.262035 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21e53c9-b252-4ba6-abb7-0c8772e2bdb6 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:54.301811 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeeae3a-c7a0-49fe-91f6-2263836c2c51 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:54.361367 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e74cf8-ca69-4e22-b33e-bd9f2c44bf01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:54.362653 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:54.363737 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e74cf8-ca69-4e22-b33e-bd9f2c44bf01 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:54.364619 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e74cf8-ca69-4e22-b33e-bd9f2c44bf01 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:54.365253 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e74cf8-ca69-4e22-b33e-bd9f2c44bf01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:54.366371 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e74cf8-ca69-4e22-b33e-bd9f2c44bf01 None admin] RBAC: Adding query filter params (role.id=d1ddee590a964ff1bbb4ee8716173021, scope.project.id=aa11ab8229b642ce962193b74001d3bc, user.id=6bfdb655a2a44b33ab724daaf73a294b) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:54.367055 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e74cf8-ca69-4e22-b33e-bd9f2c44bf01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:54.369298 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e74cf8-ca69-4e22-b33e-bd9f2c44bf01 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:54.377567 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 417/837] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 22:35:54 2022] GET /identity/v3/role_assignments?user.id=6bfdb655a2a44b33ab724daaf73a294b&scope.project.id=aa11ab8229b642ce962193b74001d3bc&role.id=d1ddee590a964ff1bbb4ee8716173021 => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:35:54.379242 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff2304-dd70-4035-8dee-fb07cf7acdc9 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:54.379669 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff2304-dd70-4035-8dee-fb07cf7acdc9 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:54.380693 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff2304-dd70-4035-8dee-fb07cf7acdc9 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:54.382453 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 418/838] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:54.388841 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20836850-b6ec-49d6-ada9-eeb990201066 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:54.389152 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20836850-b6ec-49d6-ada9-eeb990201066 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:54.389415 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20836850-b6ec-49d6-ada9-eeb990201066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:54.393313 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21e53c9-b252-4ba6-abb7-0c8772e2bdb6 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:54.410262 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-20836850-b6ec-49d6-ada9-eeb990201066 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:54.411857 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-20836850-b6ec-49d6-ada9-eeb990201066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:54.438426 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07875102-5fe4-484f-8f09-9743bd25511d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:54.439336 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07875102-5fe4-484f-8f09-9743bd25511d None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:54.439538 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07875102-5fe4-484f-8f09-9743bd25511d None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:54.439937 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07875102-5fe4-484f-8f09-9743bd25511d None admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:54.440756 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07875102-5fe4-484f-8f09-9743bd25511d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:54.441138 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07875102-5fe4-484f-8f09-9743bd25511d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:54.441315 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:54.443334 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07875102-5fe4-484f-8f09-9743bd25511d None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:54.456984 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 421/839] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:54.463821 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07875102-5fe4-484f-8f09-9743bd25511d None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:54.504545 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 419/840] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:54 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:35:54.528079 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20836850-b6ec-49d6-ada9-eeb990201066 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:54.606220 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07875102-5fe4-484f-8f09-9743bd25511d None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:54.649843 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20836850-b6ec-49d6-ada9-eeb990201066 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:54.650385 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df8e60c-7def-49e1-b074-58c27dc1211e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:54.651413 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8e60c-7def-49e1-b074-58c27dc1211e None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:54.651413 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8e60c-7def-49e1-b074-58c27dc1211e None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:54.651594 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8e60c-7def-49e1-b074-58c27dc1211e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:54.655468 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:54.661148 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df8e60c-7def-49e1-b074-58c27dc1211e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:54.663848 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df8e60c-7def-49e1-b074-58c27dc1211e None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:54.684765 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-6df8e60c-7def-49e1-b074-58c27dc1211e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 22:35:54.684765 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:54.684765 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:54.684765 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:54.684765 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:54.684765 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:54.684765 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:54.684765 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:54.684765 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:54.684765 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:54.684765 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:54.684765 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:54.684765 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:35:54.684765 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:35:54.684765 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:35:54.684765 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:35:54.686321 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:54.686321 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:54.686321 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:35:54.686321 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:35:54.686321 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:35:54.686321 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:35:54.686321 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:54.686321 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:54.686321 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:54.686321 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:54.686321 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:54.686321 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:54.686321 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:54.686321 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:54.686321 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:54.686321 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:54.686321 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:54.686321 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:54.687592 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:54.687592 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:54.687592 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:35:54.687592 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:35:54.687592 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:35:54.687592 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:35:54.687592 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:35:54.687592 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:35:54.687592 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 22:35:54.687592 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:54.687592 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 422/841] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Jul 21 22:35:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:54.692266 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9c390-ee59-4094-bbe7-b17ebfc191d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:54.693214 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9c390-ee59-4094-bbe7-b17ebfc191d5 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:54.693595 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9c390-ee59-4094-bbe7-b17ebfc191d5 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:54.693970 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9c390-ee59-4094-bbe7-b17ebfc191d5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:54.695057 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d9c390-ee59-4094-bbe7-b17ebfc191d5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:54.695914 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df8e60c-7def-49e1-b074-58c27dc1211e None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:54.696343 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d9c390-ee59-4094-bbe7-b17ebfc191d5 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:54.709250 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-93d9c390-ee59-4094-bbe7-b17ebfc191d5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 22:35:54.709250 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:54.709250 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:35:54.709250 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:35:54.709250 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:35:54.709250 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:35:54.709250 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:35:54.709250 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:35:54.709250 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 22:35:54.709250 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:54.709250 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:35:54.709250 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:54.709250 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:54.709250 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:54.709250 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:54.709250 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:54.709250 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:54.710472 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:54.710472 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:54.710472 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:54.710472 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:54.710472 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:54.710472 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:54.710472 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:35:54.710472 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:35:54.710472 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:35:54.710472 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:35:54.710472 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:54.710472 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:54.710472 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:54.710472 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:54.710472 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:54.710472 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:54.711529 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:54.711529 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:54.711529 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:54.711529 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:54.711529 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:54.711529 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:54.711529 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:54.711529 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:54.711529 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:54.711529 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:54.711529 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:35:54.711529 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:35:54.711529 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 22:35:54.711529 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:54.713469 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 420/842] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 21 22:35:54 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:35:54.719138 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9c390-ee59-4094-bbe7-b17ebfc191d5 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:54.748182 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:54.819981 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df8e60c-7def-49e1-b074-58c27dc1211e None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:54.836175 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9c390-ee59-4094-bbe7-b17ebfc191d5 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:54.857113 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dce301d-2ee4-4794-96fc-8120cbe221f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:54.857906 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce301d-2ee4-4794-96fc-8120cbe221f4 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:54.858267 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce301d-2ee4-4794-96fc-8120cbe221f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:54.859007 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce301d-2ee4-4794-96fc-8120cbe221f4 None admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:54.859671 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dce301d-2ee4-4794-96fc-8120cbe221f4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:54.860058 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dce301d-2ee4-4794-96fc-8120cbe221f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:54.861568 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dce301d-2ee4-4794-96fc-8120cbe221f4 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:54.874840 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be70a9-a713-49c5-b167-b57b60f84bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:54.875782 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be70a9-a713-49c5-b167-b57b60f84bae None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:54.876247 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be70a9-a713-49c5-b167-b57b60f84bae None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:54.876835 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be70a9-a713-49c5-b167-b57b60f84bae None admin] PATH_INFO: `/v3/services` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:54.877553 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3be70a9-a713-49c5-b167-b57b60f84bae None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:54.877872 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3be70a9-a713-49c5-b167-b57b60f84bae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:54.879248 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3be70a9-a713-49c5-b167-b57b60f84bae None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:54.883067 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 423/843] 10.222.0.52 () {56 vars in 1149 bytes} [Thu Jul 21 22:35:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:35:54.883067 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb87d94-a590-4fda-a331-c061e0c8ed4f None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:54.883067 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb87d94-a590-4fda-a331-c061e0c8ed4f None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:54.883067 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb87d94-a590-4fda-a331-c061e0c8ed4f None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:54.884613 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 424/844] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:54.886078 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb87d94-a590-4fda-a331-c061e0c8ed4f None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:54.887147 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 421/845] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 21 22:35:54 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:35:54.890714 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd7317-d0be-4355-b059-1b69f14d71ed None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:54.891180 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd7317-d0be-4355-b059-1b69f14d71ed None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:54.891629 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd7317-d0be-4355-b059-1b69f14d71ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:54.909464 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-8bcd7317-d0be-4355-b059-1b69f14d71ed None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:54.910660 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-8bcd7317-d0be-4355-b059-1b69f14d71ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:54.952301 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:54.986249 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 422/846] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:54 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:35:54.987597 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd7317-d0be-4355-b059-1b69f14d71ed None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:55.003707 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb87d94-a590-4fda-a331-c061e0c8ed4f None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:55.011576 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:55.042730 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8676e28-41a8-4362-be29-6d56697cd9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:55.043477 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8676e28-41a8-4362-be29-6d56697cd9dd None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:55.043683 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8676e28-41a8-4362-be29-6d56697cd9dd None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:55.043959 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8676e28-41a8-4362-be29-6d56697cd9dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:55.101778 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd7317-d0be-4355-b059-1b69f14d71ed None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:55.117290 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8676e28-41a8-4362-be29-6d56697cd9dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:55.118568 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8676e28-41a8-4362-be29-6d56697cd9dd None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:55.139254 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47372399-362b-441c-9423-7d8478da5a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:55.139929 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47372399-362b-441c-9423-7d8478da5a54 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:55.140219 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47372399-362b-441c-9423-7d8478da5a54 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:55.140917 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47372399-362b-441c-9423-7d8478da5a54 None admin] PATH_INFO: `/v3/services` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:55.141626 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47372399-362b-441c-9423-7d8478da5a54 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:55.141936 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47372399-362b-441c-9423-7d8478da5a54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:55.143364 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47372399-362b-441c-9423-7d8478da5a54 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:55.148290 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 423/847] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 21 22:35:54 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:55.149075 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47372399-362b-441c-9423-7d8478da5a54 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:55.191532 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:55.246834 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 425/848] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:35:54 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 22:35:55.258933 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8676e28-41a8-4362-be29-6d56697cd9dd None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:55.279178 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:55.295848 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47372399-362b-441c-9423-7d8478da5a54 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:55.344839 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127ab7d2-7cc5-4bc5-99c9-4c4dc02cc6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:55.345634 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ab7d2-7cc5-4bc5-99c9-4c4dc02cc6d9 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:55.345919 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ab7d2-7cc5-4bc5-99c9-4c4dc02cc6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:55.346197 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ab7d2-7cc5-4bc5-99c9-4c4dc02cc6d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:55.359530 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127ab7d2-7cc5-4bc5-99c9-4c4dc02cc6d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:35:55.367301 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-127ab7d2-7cc5-4bc5-99c9-4c4dc02cc6d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:35:55.367301 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:55.367301 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:55.367301 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:55.367301 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:55.367301 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:55.367301 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:55.367301 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:55.367301 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:55.367301 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:55.367301 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:55.367301 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:55.367301 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:35:55.367301 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:35:55.367301 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:35:55.367301 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:35:55.368215 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:35:55.368215 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:35:55.368215 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:35:55.368215 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:35:55.368215 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:55.368215 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:55.368215 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:55.368215 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:55.368215 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:55.368215 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:55.368215 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:55.368215 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:55.368215 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:55.368215 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:55.368215 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:55.368215 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:55.369027 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:55.369027 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:55.369027 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:55.369027 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:55.369027 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:35:55.369027 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:35:55.369027 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:35:55.369027 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:35:55.369027 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:35:55.369027 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:35:55.369027 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:35:55.369027 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:55.369027 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 424/849] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 22:35:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:55.372954 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8676e28-41a8-4362-be29-6d56697cd9dd None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:55.374216 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127ab7d2-7cc5-4bc5-99c9-4c4dc02cc6d9 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:55.409894 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecbb463-c0c5-4380-980c-337b16d32bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:55.410924 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecbb463-c0c5-4380-980c-337b16d32bf4 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:55.411306 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecbb463-c0c5-4380-980c-337b16d32bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:55.411759 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecbb463-c0c5-4380-980c-337b16d32bf4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:55.422509 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecbb463-c0c5-4380-980c-337b16d32bf4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:55.424190 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecbb463-c0c5-4380-980c-337b16d32bf4 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:55.444537 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-6ecbb463-c0c5-4380-980c-337b16d32bf4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:55.444537 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:55.444537 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:55.444537 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:55.444537 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:55.444537 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:55.444537 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:55.444537 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:55.444537 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:55.444537 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:55.444537 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:55.444537 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:55.444537 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:55.444537 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:55.444537 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:55.445624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:55.445624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:55.445624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:55.445624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:55.445624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:55.445624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:55.445624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:55.445624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:55.445624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:55.445624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:55.445624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:55.445624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:55.445624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:55.445624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:55.445624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:55.445624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:55.445624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:55.446689 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:55.446689 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:55.446689 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:55.446689 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:55.447612 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 426/850] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:55.452717 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecbb463-c0c5-4380-980c-337b16d32bf4 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:55.462735 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:55.501086 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127ab7d2-7cc5-4bc5-99c9-4c4dc02cc6d9 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:55.542948 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed9009-46ba-4139-a883-234797c79ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:55.542948 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed9009-46ba-4139-a883-234797c79ac1 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:55.542948 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed9009-46ba-4139-a883-234797c79ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:55.542948 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed9009-46ba-4139-a883-234797c79ac1 None admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:55.543410 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fed9009-46ba-4139-a883-234797c79ac1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:55.543410 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fed9009-46ba-4139-a883-234797c79ac1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:55.545122 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fed9009-46ba-4139-a883-234797c79ac1 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:55.554951 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 425/851] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:35:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:55.562967 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed9009-46ba-4139-a883-234797c79ac1 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:55.564294 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:55.572172 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecbb463-c0c5-4380-980c-337b16d32bf4 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:55.618613 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89659d60-36ae-4ee5-8d7a-982ea36736bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:55.619635 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89659d60-36ae-4ee5-8d7a-982ea36736bd None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:55.619932 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89659d60-36ae-4ee5-8d7a-982ea36736bd None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:55.620216 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89659d60-36ae-4ee5-8d7a-982ea36736bd None admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:55.620956 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89659d60-36ae-4ee5-8d7a-982ea36736bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:55.621268 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89659d60-36ae-4ee5-8d7a-982ea36736bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:55.623228 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89659d60-36ae-4ee5-8d7a-982ea36736bd None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:55.642906 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 427/852] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:35:55.643834 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89659d60-36ae-4ee5-8d7a-982ea36736bd None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:55.694179 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed9009-46ba-4139-a883-234797c79ac1 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:55.716123 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:55.740623 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce90d8-0c20-4f20-8174-fdccb4f60831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:55.742043 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce90d8-0c20-4f20-8174-fdccb4f60831 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:55.742673 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce90d8-0c20-4f20-8174-fdccb4f60831 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:55.743479 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce90d8-0c20-4f20-8174-fdccb4f60831 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:55.753040 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89659d60-36ae-4ee5-8d7a-982ea36736bd None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:55.759453 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bce90d8-0c20-4f20-8174-fdccb4f60831 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:55.761590 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bce90d8-0c20-4f20-8174-fdccb4f60831 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:55.785370 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-2bce90d8-0c20-4f20-8174-fdccb4f60831 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:55.785370 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:55.785370 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:55.785370 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:55.785370 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:55.785370 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:55.785370 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:55.785370 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:55.785370 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:55.785370 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:55.785370 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:55.785370 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:55.785370 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:55.785370 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:55.785370 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:55.787193 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:55.787193 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:55.787193 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:55.787193 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:55.787193 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:55.787193 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:55.787193 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:55.787193 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:55.787193 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:55.787193 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:55.787193 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:55.787193 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:55.787193 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:55.787193 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:55.787193 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:55.787193 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:55.787193 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:55.788097 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:55.788097 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:55.788097 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:55.788097 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:55.788097 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 426/853] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:55.795196 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce90d8-0c20-4f20-8174-fdccb4f60831 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:55.810341 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f14d17-3fc6-4cef-88d4-1852c56667d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:55.811333 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f14d17-3fc6-4cef-88d4-1852c56667d1 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:55.811767 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f14d17-3fc6-4cef-88d4-1852c56667d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:55.812291 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f14d17-3fc6-4cef-88d4-1852c56667d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:55.820728 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f14d17-3fc6-4cef-88d4-1852c56667d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:55.822265 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f14d17-3fc6-4cef-88d4-1852c56667d1 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:55.843985 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-71f14d17-3fc6-4cef-88d4-1852c56667d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:35:55.843985 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:55.843985 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:55.843985 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:55.843985 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:55.843985 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:55.843985 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:55.843985 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:55.843985 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:55.843985 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:55.843985 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:55.843985 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:55.843985 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:35:55.843985 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:35:55.843985 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:35:55.844926 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:35:55.844926 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:55.844926 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:55.844926 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:55.844926 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:55.844926 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:55.844926 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:55.844926 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:55.844926 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:55.844926 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:55.844926 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:55.844926 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:55.844926 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:55.844926 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:55.844926 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:55.844926 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:55.844926 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:55.845809 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:35:55.845809 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:35:55.845809 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:35:55.845809 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:35:55.845809 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:35:55.845809 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:35:55.845809 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:35:55.845809 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:55.845809 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 428/854] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:35:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:55.847916 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f14d17-3fc6-4cef-88d4-1852c56667d1 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:55.860295 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:55.907448 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce90d8-0c20-4f20-8174-fdccb4f60831 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:55.957683 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ad408-c0e7-410e-a3f4-ecd02fea336a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:55.959967 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ad408-c0e7-410e-a3f4-ecd02fea336a None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:55.960471 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ad408-c0e7-410e-a3f4-ecd02fea336a None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:55.961090 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ad408-c0e7-410e-a3f4-ecd02fea336a None admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:55.962326 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59ad408-c0e7-410e-a3f4-ecd02fea336a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:55.963086 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59ad408-c0e7-410e-a3f4-ecd02fea336a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:55.965156 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59ad408-c0e7-410e-a3f4-ecd02fea336a None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:55.968232 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f14d17-3fc6-4cef-88d4-1852c56667d1 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:55.980754 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 427/855] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:35:55.988414 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ad408-c0e7-410e-a3f4-ecd02fea336a None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:55.999548 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:56.017852 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ce96e3-59af-4833-abc8-7aa287018e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:56.018553 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce96e3-59af-4833-abc8-7aa287018e21 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:56.018868 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce96e3-59af-4833-abc8-7aa287018e21 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:56.019126 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce96e3-59af-4833-abc8-7aa287018e21 None admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:56.019916 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ce96e3-59af-4833-abc8-7aa287018e21 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:56.020223 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ce96e3-59af-4833-abc8-7aa287018e21 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:56.022222 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ce96e3-59af-4833-abc8-7aa287018e21 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:56.035028 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 429/856] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 22:35:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:56.040518 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ce96e3-59af-4833-abc8-7aa287018e21 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:56.124590 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ad408-c0e7-410e-a3f4-ecd02fea336a None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:56.148070 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:56.171257 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3825a0-1782-4e0c-bfc7-fc4aacd22c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:56.171783 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ce96e3-59af-4833-abc8-7aa287018e21 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:56.172472 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3825a0-1782-4e0c-bfc7-fc4aacd22c38 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:56.172901 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3825a0-1782-4e0c-bfc7-fc4aacd22c38 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:56.173320 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3825a0-1782-4e0c-bfc7-fc4aacd22c38 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:56.183653 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3825a0-1782-4e0c-bfc7-fc4aacd22c38 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:56.185807 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3825a0-1782-4e0c-bfc7-fc4aacd22c38 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:56.207719 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-0b3825a0-1782-4e0c-bfc7-fc4aacd22c38 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 22:35:56.207719 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:56.207719 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:56.207719 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:56.207719 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:56.207719 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:56.207719 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:56.207719 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:56.207719 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:56.207719 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:56.207719 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:56.207719 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:56.207719 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:35:56.207719 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:35:56.207719 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:35:56.207719 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:35:56.208971 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:56.208971 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:56.208971 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:35:56.208971 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:35:56.208971 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:35:56.208971 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:35:56.208971 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:56.208971 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:56.208971 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:56.208971 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:56.208971 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:56.208971 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:56.208971 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:56.208971 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:56.208971 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:56.208971 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:56.208971 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:56.208971 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:56.210102 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:56.210102 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:56.210102 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:35:56.210102 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:35:56.210102 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:35:56.210102 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:35:56.210102 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:35:56.210102 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:35:56.210102 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 22:35:56.210102 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:56.213176 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 428/857] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 22:35:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:56.218971 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3825a0-1782-4e0c-bfc7-fc4aacd22c38 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:56.219740 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650e5f06-77bd-4d5a-9495-abf5397a0b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:56.220677 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e5f06-77bd-4d5a-9495-abf5397a0b30 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:56.221063 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e5f06-77bd-4d5a-9495-abf5397a0b30 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:56.221439 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e5f06-77bd-4d5a-9495-abf5397a0b30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:56.222402 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650e5f06-77bd-4d5a-9495-abf5397a0b30 None admin] RBAC: Adding query filter params (role.id=715d866890d14923b11470d544b4f960, scope.project.id=aa11ab8229b642ce962193b74001d3bc, user.id=63cc904578554aea84ff1917a5ed075e) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:56.222974 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650e5f06-77bd-4d5a-9495-abf5397a0b30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:56.225371 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650e5f06-77bd-4d5a-9495-abf5397a0b30 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:56.233188 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 430/858] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 22:35:56 2022] GET /identity/v3/role_assignments?user.id=63cc904578554aea84ff1917a5ed075e&scope.project.id=aa11ab8229b642ce962193b74001d3bc&role.id=715d866890d14923b11470d544b4f960 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:56.345488 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3825a0-1782-4e0c-bfc7-fc4aacd22c38 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:56.387759 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53bc048-a0c8-4185-8b22-036a11428ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:56.388546 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53bc048-a0c8-4185-8b22-036a11428ca0 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:56.388931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53bc048-a0c8-4185-8b22-036a11428ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:56.389526 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53bc048-a0c8-4185-8b22-036a11428ca0 None admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:56.391174 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53bc048-a0c8-4185-8b22-036a11428ca0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:56.391589 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53bc048-a0c8-4185-8b22-036a11428ca0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:56.393429 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53bc048-a0c8-4185-8b22-036a11428ca0 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:56.419505 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 429/859] 10.222.0.52 () {56 vars in 1135 bytes} [Thu Jul 21 22:35:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:35:56.425455 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650e5f06-77bd-4d5a-9495-abf5397a0b30 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:56.490016 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:56.548736 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650e5f06-77bd-4d5a-9495-abf5397a0b30 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:56.585040 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fb947e-b42f-4cd3-b8b9-515b3326218c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:56.585846 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb947e-b42f-4cd3-b8b9-515b3326218c None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:56.586202 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb947e-b42f-4cd3-b8b9-515b3326218c None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:56.586552 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb947e-b42f-4cd3-b8b9-515b3326218c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:56.618084 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2305978-7f18-4648-ac28-86274fb54eeb None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:56.618281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2305978-7f18-4648-ac28-86274fb54eeb None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:56.618605 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2305978-7f18-4648-ac28-86274fb54eeb None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:56.619999 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 430/860] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:35:56.626764 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b759c38-737c-4287-b921-72f2447189f8 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:56.626764 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b759c38-737c-4287-b921-72f2447189f8 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:56.627032 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b759c38-737c-4287-b921-72f2447189f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:56.648844 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-4b759c38-737c-4287-b921-72f2447189f8 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:56.650224 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-4b759c38-737c-4287-b921-72f2447189f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:56.667756 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fb947e-b42f-4cd3-b8b9-515b3326218c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:56.670242 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fb947e-b42f-4cd3-b8b9-515b3326218c None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:56.681643 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:56.734801 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 431/861] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:56 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:35:56.737527 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:56.754545 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b759c38-737c-4287-b921-72f2447189f8 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:56.799128 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 431/862] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:35:56 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 22:35:56.801159 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fb947e-b42f-4cd3-b8b9-515b3326218c None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:56.884554 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b759c38-737c-4287-b921-72f2447189f8 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:56.908711 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fb947e-b42f-4cd3-b8b9-515b3326218c None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:56.928890 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d8dd6-91c4-4d38-ade8-05199c6e4c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:56.930014 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d8dd6-91c4-4d38-ade8-05199c6e4c5f None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:56.930466 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d8dd6-91c4-4d38-ade8-05199c6e4c5f None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:56.931082 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d8dd6-91c4-4d38-ade8-05199c6e4c5f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:56.931899 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d8dd6-91c4-4d38-ade8-05199c6e4c5f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:56.933403 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d8dd6-91c4-4d38-ade8-05199c6e4c5f None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:56.947047 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-d38d8dd6-91c4-4d38-ade8-05199c6e4c5f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 22:35:56.947047 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:56.947047 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:35:56.947047 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:35:56.947047 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:35:56.947047 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:35:56.947047 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:35:56.947047 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:35:56.947047 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 22:35:56.947047 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:56.947047 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:35:56.947047 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:56.947047 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:56.947047 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:56.947047 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:56.947047 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:56.948201 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:56.948201 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:56.948201 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:56.948201 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:56.948201 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:56.948201 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:56.948201 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:56.948201 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:35:56.948201 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:35:56.948201 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:35:56.948201 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:35:56.948201 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:56.948201 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:56.948201 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:56.948201 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:56.948201 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:56.949268 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:56.949268 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:56.949268 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:56.949268 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:56.949268 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:56.949268 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:56.949268 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:56.949268 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:56.949268 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:56.949268 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:56.949268 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:56.949268 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:35:56.949268 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:35:56.949268 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 22:35:56.949268 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:56.956035 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d7a9d0-6f49-440c-aa33-1b0e910fdae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:56.956035 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d7a9d0-6f49-440c-aa33-1b0e910fdae4 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:56.956035 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d7a9d0-6f49-440c-aa33-1b0e910fdae4 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:56.956035 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d7a9d0-6f49-440c-aa33-1b0e910fdae4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:56.956833 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 432/863] 10.222.0.52 () {56 vars in 1132 bytes} [Thu Jul 21 22:35:56 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:35:56.957865 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d8dd6-91c4-4d38-ade8-05199c6e4c5f None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:56.966905 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d7a9d0-6f49-440c-aa33-1b0e910fdae4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:56.969014 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d7a9d0-6f49-440c-aa33-1b0e910fdae4 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:56.989801 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:56.995277 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-96d7a9d0-6f49-440c-aa33-1b0e910fdae4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:56.995277 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:56.995277 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:56.995277 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:56.995277 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:56.995277 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:56.995277 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:56.995277 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:56.995277 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:56.995277 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:56.995277 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:56.995277 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:56.995277 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:56.995277 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:56.995277 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:57.002145 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:57.002145 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:57.002145 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:57.002145 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:57.002145 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:57.002145 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:57.002145 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:57.002145 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:57.002145 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:57.002145 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:57.002145 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:57.002145 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:57.002145 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:57.002145 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:57.002145 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:57.002145 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:57.002145 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:57.003406 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:57.003406 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:57.003406 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:57.003406 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:57.003406 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 432/864] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:35:57.003406 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d7a9d0-6f49-440c-aa33-1b0e910fdae4 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:57.009762 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:57.084009 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d8dd6-91c4-4d38-ade8-05199c6e4c5f None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:57.119261 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2242f43-f64c-4944-b5ed-560b5a7275af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:57.120277 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2242f43-f64c-4944-b5ed-560b5a7275af None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:57.120794 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2242f43-f64c-4944-b5ed-560b5a7275af None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:57.121266 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2242f43-f64c-4944-b5ed-560b5a7275af None admin] PATH_INFO: `/v3/services` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:57.122183 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2242f43-f64c-4944-b5ed-560b5a7275af None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:57.122687 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2242f43-f64c-4944-b5ed-560b5a7275af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:57.124277 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2242f43-f64c-4944-b5ed-560b5a7275af None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:57.134047 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d7a9d0-6f49-440c-aa33-1b0e910fdae4 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:57.134846 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 433/865] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Jul 21 22:35:56 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:57.135959 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2242f43-f64c-4944-b5ed-560b5a7275af None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:57.169906 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d0553e-34c9-46a6-820e-e28c36c7fe4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:57.170495 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0553e-34c9-46a6-820e-e28c36c7fe4c None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:57.170876 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0553e-34c9-46a6-820e-e28c36c7fe4c None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:57.171161 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0553e-34c9-46a6-820e-e28c36c7fe4c None admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:57.171947 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d0553e-34c9-46a6-820e-e28c36c7fe4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:57.172311 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d0553e-34c9-46a6-820e-e28c36c7fe4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:57.174062 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d0553e-34c9-46a6-820e-e28c36c7fe4c None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:57.190807 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 433/866] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:57.194614 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d0553e-34c9-46a6-820e-e28c36c7fe4c None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:57.239144 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:57.249448 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2242f43-f64c-4944-b5ed-560b5a7275af None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:57.262261 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:57.287316 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83fddec-a135-477e-b7ce-dbaee34d7e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:57.288305 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83fddec-a135-477e-b7ce-dbaee34d7e39 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:57.288771 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83fddec-a135-477e-b7ce-dbaee34d7e39 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:57.289243 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83fddec-a135-477e-b7ce-dbaee34d7e39 None admin] PATH_INFO: `/v3/services` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:57.290134 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83fddec-a135-477e-b7ce-dbaee34d7e39 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:57.290617 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83fddec-a135-477e-b7ce-dbaee34d7e39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:57.292223 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83fddec-a135-477e-b7ce-dbaee34d7e39 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:57.302800 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 434/867] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Jul 21 22:35:57 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:57.302800 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dece12aa-bf8a-4183-9dd6-27d1aa1297e5 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:57.302800 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dece12aa-bf8a-4183-9dd6-27d1aa1297e5 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:57.302800 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dece12aa-bf8a-4183-9dd6-27d1aa1297e5 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:57.305452 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d0553e-34c9-46a6-820e-e28c36c7fe4c None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:57.308046 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 435/868] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:57 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:57.310450 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab3e0e-0f0a-4078-a9cb-5af28e6c3aff None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:57.310741 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab3e0e-0f0a-4078-a9cb-5af28e6c3aff None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:57.311049 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab3e0e-0f0a-4078-a9cb-5af28e6c3aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:57.336520 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-e6ab3e0e-0f0a-4078-a9cb-5af28e6c3aff None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:57.338523 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-e6ab3e0e-0f0a-4078-a9cb-5af28e6c3aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:57.349733 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325ba340-e0b1-46fd-b7a1-168ca9d62cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:57.350544 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ba340-e0b1-46fd-b7a1-168ca9d62cdf None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:57.350852 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ba340-e0b1-46fd-b7a1-168ca9d62cdf None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:57.351557 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ba340-e0b1-46fd-b7a1-168ca9d62cdf None admin] PATH_INFO: `/v3/projects/service` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:57.360722 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325ba340-e0b1-46fd-b7a1-168ca9d62cdf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:57.362205 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325ba340-e0b1-46fd-b7a1-168ca9d62cdf None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:57.388404 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-325ba340-e0b1-46fd-b7a1-168ca9d62cdf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:35:57.388404 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:57.388404 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:57.388404 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:57.388404 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:57.388404 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:57.388404 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:57.388404 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:57.388404 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:57.388404 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:57.388404 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:57.388404 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:57.388404 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:35:57.388404 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:35:57.388404 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:35:57.389363 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:35:57.389363 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:57.389363 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:57.389363 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:57.389363 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:57.389363 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:57.389363 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:57.389363 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:57.389363 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:57.389363 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:57.389363 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:57.389363 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:57.389363 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:57.389363 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:57.389363 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:57.389363 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:57.389363 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:57.390522 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:35:57.390522 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:35:57.390522 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:35:57.390522 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:35:57.390522 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:35:57.390522 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:35:57.390522 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:35:57.390522 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:35:57.390522 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 434/869] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:35:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:35:57.396296 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325ba340-e0b1-46fd-b7a1-168ca9d62cdf None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:57.437036 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 436/870] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:57 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:35:57.460687 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ab3e0e-0f0a-4078-a9cb-5af28e6c3aff None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:57.549491 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325ba340-e0b1-46fd-b7a1-168ca9d62cdf None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:57.565942 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:57.585149 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ab3e0e-0f0a-4078-a9cb-5af28e6c3aff None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:57.585965 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:57.600559 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde48053-8023-421a-acae-e83e8579a3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:57.601688 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde48053-8023-421a-acae-e83e8579a3ff None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:57.601974 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde48053-8023-421a-acae-e83e8579a3ff None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:57.602245 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde48053-8023-421a-acae-e83e8579a3ff None admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:57.603464 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde48053-8023-421a-acae-e83e8579a3ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:57.603777 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde48053-8023-421a-acae-e83e8579a3ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:57.605748 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde48053-8023-421a-acae-e83e8579a3ff None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:57.625850 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 435/871] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 22:35:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:35:57.627715 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde48053-8023-421a-acae-e83e8579a3ff None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:57.629671 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4185b8ae-7718-4e06-b98c-e6c02f7d7f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:57.630592 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4185b8ae-7718-4e06-b98c-e6c02f7d7f01 None admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:57.631096 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4185b8ae-7718-4e06-b98c-e6c02f7d7f01 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:57.631579 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4185b8ae-7718-4e06-b98c-e6c02f7d7f01 None admin] PATH_INFO: `/v3/services` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:57.632505 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4185b8ae-7718-4e06-b98c-e6c02f7d7f01 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:57.634093 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4185b8ae-7718-4e06-b98c-e6c02f7d7f01 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:57.707720 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 437/872] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jul 21 22:35:57 2022] POST /identity/v3/services => generated 251 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:35:57.743902 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde48053-8023-421a-acae-e83e8579a3ff None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:57.795379 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6be8938-1674-45ee-80f6-f74dc3207dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:57.796050 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6be8938-1674-45ee-80f6-f74dc3207dfc None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:57.796328 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6be8938-1674-45ee-80f6-f74dc3207dfc None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:57.796610 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6be8938-1674-45ee-80f6-f74dc3207dfc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:57.797461 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6be8938-1674-45ee-80f6-f74dc3207dfc None admin] RBAC: Adding query filter params (role.id=d1ddee590a964ff1bbb4ee8716173021, scope.project.id=aa11ab8229b642ce962193b74001d3bc, user.id=ba7eef4e1b27476ab4016449d9c3a4b9) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:57.797774 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6be8938-1674-45ee-80f6-f74dc3207dfc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:57.799256 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6be8938-1674-45ee-80f6-f74dc3207dfc None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:57.807302 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 436/873] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 22:35:57 2022] GET /identity/v3/role_assignments?user.id=ba7eef4e1b27476ab4016449d9c3a4b9&scope.project.id=aa11ab8229b642ce962193b74001d3bc&role.id=d1ddee590a964ff1bbb4ee8716173021 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:58.511546 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a7840-50ea-43f9-b802-65457c8dbe1b None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:58.512013 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a7840-50ea-43f9-b802-65457c8dbe1b None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:58.512093 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a7840-50ea-43f9-b802-65457c8dbe1b None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:58.513721 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 438/874] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:58.518753 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3251f94-a918-47e5-b8da-926b1f26ca95 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:58.519065 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3251f94-a918-47e5-b8da-926b1f26ca95 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:58.519346 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3251f94-a918-47e5-b8da-926b1f26ca95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:58.536686 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-a3251f94-a918-47e5-b8da-926b1f26ca95 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:58.537889 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-a3251f94-a918-47e5-b8da-926b1f26ca95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:58.576389 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:58.609065 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 437/875] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:58 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:35:58.633838 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a7840-50ea-43f9-b802-65457c8dbe1b None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:58.741234 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a7840-50ea-43f9-b802-65457c8dbe1b None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:58.775653 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9799fa-9df1-4c77-9f00-c9b5d2b24f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:58.776389 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9799fa-9df1-4c77-9f00-c9b5d2b24f94 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:58.776669 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9799fa-9df1-4c77-9f00-c9b5d2b24f94 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:58.776943 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9799fa-9df1-4c77-9f00-c9b5d2b24f94 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:58.788209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9799fa-9df1-4c77-9f00-c9b5d2b24f94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:35:58.796140 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-cd9799fa-9df1-4c77-9f00-c9b5d2b24f94 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 22:35:58.796140 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:58.796140 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:58.796140 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:58.796140 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:58.796140 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:58.796140 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:58.796140 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:58.796140 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:58.796140 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:58.796140 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:58.796140 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:58.796140 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:35:58.796140 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:35:58.796140 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:35:58.796140 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:35:58.797087 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:35:58.797087 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:35:58.797087 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:35:58.797087 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:35:58.797087 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:58.797087 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:58.797087 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:58.797087 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:58.797087 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:58.797087 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:58.797087 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:58.797087 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:58.797087 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:58.797087 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:58.797087 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:58.797087 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:58.797931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:58.797931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:58.797931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:58.797931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:58.797931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:35:58.797931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:35:58.797931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:35:58.797931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:35:58.797931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:35:58.797931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:35:58.797931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 22:35:58.797931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:58.797931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 439/876] 10.222.0.52 () {56 vars in 1120 bytes} [Thu Jul 21 22:35:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:58.803823 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3251f94-a918-47e5-b8da-926b1f26ca95 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:58.912868 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3251f94-a918-47e5-b8da-926b1f26ca95 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:58.949742 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333fe5ad-8d70-4a04-8ffb-b0ceda6efbfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:58.950418 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333fe5ad-8d70-4a04-8ffb-b0ceda6efbfb None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:58.950701 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333fe5ad-8d70-4a04-8ffb-b0ceda6efbfb None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:58.951003 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333fe5ad-8d70-4a04-8ffb-b0ceda6efbfb None admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:58.951752 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333fe5ad-8d70-4a04-8ffb-b0ceda6efbfb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:58.952058 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333fe5ad-8d70-4a04-8ffb-b0ceda6efbfb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:58.953423 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333fe5ad-8d70-4a04-8ffb-b0ceda6efbfb None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:58.963049 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 438/877] 10.222.0.52 () {56 vars in 1115 bytes} [Thu Jul 21 22:35:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:58.968741 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9799fa-9df1-4c77-9f00-c9b5d2b24f94 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:58.981923 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:59.075707 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9799fa-9df1-4c77-9f00-c9b5d2b24f94 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:59.112249 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cac9e6-6a94-4334-8e58-83f8b9625cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:59.113043 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cac9e6-6a94-4334-8e58-83f8b9625cfc None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:59.113324 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cac9e6-6a94-4334-8e58-83f8b9625cfc None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:59.113601 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cac9e6-6a94-4334-8e58-83f8b9625cfc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:59.123961 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cac9e6-6a94-4334-8e58-83f8b9625cfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:59.125393 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cac9e6-6a94-4334-8e58-83f8b9625cfc None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:59.143161 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-32cac9e6-6a94-4334-8e58-83f8b9625cfc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:59.143161 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:59.143161 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:59.143161 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:59.143161 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:59.143161 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:59.143161 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:59.143161 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:59.143161 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:59.143161 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:59.143161 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:59.143161 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:59.143161 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:35:59.143161 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:35:59.143161 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:35:59.144989 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:35:59.144989 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:59.144989 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:59.144989 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:59.144989 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:59.144989 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:59.144989 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:59.144989 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:59.144989 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:59.144989 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:59.144989 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:59.144989 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:59.144989 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:59.144989 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:59.144989 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:59.144989 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:59.144989 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:59.146990 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:35:59.146990 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:35:59.146990 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:35:59.146990 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:59.146990 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 440/878] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:35:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:59.150533 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333fe5ad-8d70-4a04-8ffb-b0ceda6efbfb None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:59.223222 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:59.258933 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333fe5ad-8d70-4a04-8ffb-b0ceda6efbfb None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:59.295046 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f50575-ea6f-45dc-94d4-36f5efa72810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:59.295717 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f50575-ea6f-45dc-94d4-36f5efa72810 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:59.295998 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f50575-ea6f-45dc-94d4-36f5efa72810 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:59.296271 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f50575-ea6f-45dc-94d4-36f5efa72810 None admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:59.297055 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f50575-ea6f-45dc-94d4-36f5efa72810 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:59.297363 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f50575-ea6f-45dc-94d4-36f5efa72810 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:59.298788 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f50575-ea6f-45dc-94d4-36f5efa72810 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:59.311328 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 439/879] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:35:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:35:59.316746 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cac9e6-6a94-4334-8e58-83f8b9625cfc None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:59.377242 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:59.420132 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cac9e6-6a94-4334-8e58-83f8b9625cfc None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:59.449003 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f8d6f-8f28-468d-bc82-7bdd7a6f1e6f None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:59.449680 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f8d6f-8f28-468d-bc82-7bdd7a6f1e6f None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:59.450228 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f8d6f-8f28-468d-bc82-7bdd7a6f1e6f None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:59.451932 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 440/880] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:59.455151 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d2c71-fffb-43be-b82c-83f5bf3601d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:59.455979 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d2c71-fffb-43be-b82c-83f5bf3601d7 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:59.456452 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d2c71-fffb-43be-b82c-83f5bf3601d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:59.456642 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d2c71-fffb-43be-b82c-83f5bf3601d7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:59.458493 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e13aaf-b5ca-4128-80b5-5e5f0bd5d03b None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:59.459108 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e13aaf-b5ca-4128-80b5-5e5f0bd5d03b None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:59.459644 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e13aaf-b5ca-4128-80b5-5e5f0bd5d03b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:59.465390 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167d2c71-fffb-43be-b82c-83f5bf3601d7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:59.466999 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167d2c71-fffb-43be-b82c-83f5bf3601d7 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:59.482632 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-167d2c71-fffb-43be-b82c-83f5bf3601d7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 22:35:59.482632 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:35:59.482632 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:35:59.482632 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:35:59.482632 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:35:59.482632 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:35:59.482632 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:35:59.482632 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:35:59.482632 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:35:59.482632 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:35:59.482632 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:35:59.482632 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:35:59.482632 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:35:59.482632 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:35:59.482632 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:35:59.482632 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:35:59.483650 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:35:59.483650 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:35:59.483650 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:35:59.483650 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:35:59.483650 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:35:59.483650 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:35:59.483650 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:35:59.483650 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:35:59.483650 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:35:59.483650 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:35:59.483650 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:35:59.483650 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:35:59.483650 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:35:59.483650 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:35:59.483650 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:35:59.483650 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:35:59.483650 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:35:59.483650 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:35:59.484606 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:35:59.484606 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:35:59.484606 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:35:59.484606 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:35:59.484606 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:35:59.484606 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:35:59.484606 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:35:59.484606 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:35:59.484606 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 22:35:59.484606 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:35:59.484606 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 441/881] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Jul 21 22:35:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:35:59.490004 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-31e13aaf-b5ca-4128-80b5-5e5f0bd5d03b None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:59.490004 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-31e13aaf-b5ca-4128-80b5-5e5f0bd5d03b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:35:59.495030 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d2c71-fffb-43be-b82c-83f5bf3601d7 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:59.582821 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 441/882] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:59 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:35:59.605052 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e13aaf-b5ca-4128-80b5-5e5f0bd5d03b None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:59.633314 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:59.635143 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d2c71-fffb-43be-b82c-83f5bf3601d7 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:59.658180 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:59.683701 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47abd56-bb51-4ddd-b9d2-4636b02040fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:59.684369 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47abd56-bb51-4ddd-b9d2-4636b02040fb None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:59.684659 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47abd56-bb51-4ddd-b9d2-4636b02040fb None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:59.684979 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47abd56-bb51-4ddd-b9d2-4636b02040fb None admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:59.685763 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47abd56-bb51-4ddd-b9d2-4636b02040fb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:35:59.686066 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47abd56-bb51-4ddd-b9d2-4636b02040fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:59.687586 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47abd56-bb51-4ddd-b9d2-4636b02040fb None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:59.719498 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 442/883] 10.222.0.52 () {56 vars in 1149 bytes} [Thu Jul 21 22:35:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:35:59.720666 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47abd56-bb51-4ddd-b9d2-4636b02040fb None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:35:59.731309 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e13aaf-b5ca-4128-80b5-5e5f0bd5d03b None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:59.778538 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cae256-9254-4a3d-bb2d-c3b897616812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:59.779613 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cae256-9254-4a3d-bb2d-c3b897616812 None admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:59.780083 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cae256-9254-4a3d-bb2d-c3b897616812 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:59.780554 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cae256-9254-4a3d-bb2d-c3b897616812 None admin] PATH_INFO: `/v3/services` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:59.781483 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cae256-9254-4a3d-bb2d-c3b897616812 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:59.783077 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cae256-9254-4a3d-bb2d-c3b897616812 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:59.826305 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47abd56-bb51-4ddd-b9d2-4636b02040fb None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:35:59.859576 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9b5d5e-b925-4410-9dfb-b9dfb7b03989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:35:59.860237 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b5d5e-b925-4410-9dfb-b9dfb7b03989 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:59.860514 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b5d5e-b925-4410-9dfb-b9dfb7b03989 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:59.860786 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b5d5e-b925-4410-9dfb-b9dfb7b03989 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:59.905623 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:35:59.915763 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 442/884] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jul 21 22:35:59 2022] POST /identity/v3/services => generated 269 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:35:59.917676 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8de193-1727-40b6-8fed-d07f7bef1042 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:59.917960 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8de193-1727-40b6-8fed-d07f7bef1042 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:59.918223 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8de193-1727-40b6-8fed-d07f7bef1042 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:59.919696 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 443/885] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:35:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:35:59.925479 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c64cd9d-326d-4caf-9c42-8b07b5ea475a None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:35:59.925779 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c64cd9d-326d-4caf-9c42-8b07b5ea475a None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:35:59.926045 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c64cd9d-326d-4caf-9c42-8b07b5ea475a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:35:59.940751 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9b5d5e-b925-4410-9dfb-b9dfb7b03989 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:35:59.942728 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9b5d5e-b925-4410-9dfb-b9dfb7b03989 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:35:59.944596 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-7c64cd9d-326d-4caf-9c42-8b07b5ea475a None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:35:59.945868 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-7c64cd9d-326d-4caf-9c42-8b07b5ea475a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:00.039221 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 444/886] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:35:59 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:36:00.040511 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4883123b-c5e5-4164-ab5b-18f686f1dab4 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:00.040807 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4883123b-c5e5-4164-ab5b-18f686f1dab4 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:00.041083 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4883123b-c5e5-4164-ab5b-18f686f1dab4 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:00.046532 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 445/887] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:36:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:36:00.048325 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b2fac-489c-4ccb-96b4-91665976aa18 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:00.048619 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b2fac-489c-4ccb-96b4-91665976aa18 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:00.048912 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b2fac-489c-4ccb-96b4-91665976aa18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:00.070381 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-d14b2fac-489c-4ccb-96b4-91665976aa18 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:00.071644 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-d14b2fac-489c-4ccb-96b4-91665976aa18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:00.074728 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:00.081517 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 443/888] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:35:59 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 22:36:00.081517 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9b5d5e-b925-4410-9dfb-b9dfb7b03989 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:00.156939 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 446/889] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:36:00 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:36:00.157784 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b2fac-489c-4ccb-96b4-91665976aa18 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:00.172216 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:00.199538 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9b5d5e-b925-4410-9dfb-b9dfb7b03989 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:00.237924 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da690502-2350-40ca-ae46-61f056dd37ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:00.238723 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da690502-2350-40ca-ae46-61f056dd37ee None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:00.239473 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da690502-2350-40ca-ae46-61f056dd37ee None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:00.239755 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da690502-2350-40ca-ae46-61f056dd37ee None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:00.240417 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da690502-2350-40ca-ae46-61f056dd37ee None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:00.242668 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da690502-2350-40ca-ae46-61f056dd37ee None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:00.258596 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-da690502-2350-40ca-ae46-61f056dd37ee None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 22:36:00.258596 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:00.258596 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:36:00.258596 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:36:00.258596 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:36:00.258596 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:36:00.258596 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:36:00.258596 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:36:00.258596 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 22:36:00.258596 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:36:00.258596 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:36:00.258596 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:36:00.258596 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:00.258596 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:00.258596 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:00.258596 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:00.258596 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:00.259818 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:00.259818 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:00.259818 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:00.259818 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:00.259818 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:00.259818 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:00.259818 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:36:00.259818 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:36:00.259818 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:36:00.259818 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:36:00.259818 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:00.259818 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:00.259818 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:36:00.259818 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:36:00.259818 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:36:00.259818 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:36:00.265691 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:36:00.265691 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:36:00.265691 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:36:00.265691 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:36:00.265691 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:36:00.265691 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:36:00.265691 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:36:00.265691 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:36:00.265691 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:36:00.265691 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:36:00.265691 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:36:00.265691 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:36:00.265691 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 22:36:00.265691 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:36:00.265691 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 444/890] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 21 22:36:00 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:36:00.266678 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da690502-2350-40ca-ae46-61f056dd37ee None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:00.266765 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b2fac-489c-4ccb-96b4-91665976aa18 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:00.302181 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbd642b-2db3-4a29-b261-7eb63d5a906c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:00.303008 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbd642b-2db3-4a29-b261-7eb63d5a906c None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:00.303292 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbd642b-2db3-4a29-b261-7eb63d5a906c None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:00.303569 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbd642b-2db3-4a29-b261-7eb63d5a906c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:00.312509 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:00.315230 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbd642b-2db3-4a29-b261-7eb63d5a906c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:00.316845 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbd642b-2db3-4a29-b261-7eb63d5a906c None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:00.335593 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-fcbd642b-2db3-4a29-b261-7eb63d5a906c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:36:00.335593 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:00.335593 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:00.335593 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:00.335593 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:00.335593 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:00.335593 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:00.335593 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:00.335593 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:00.335593 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:00.335593 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:00.335593 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:00.335593 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:36:00.335593 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:36:00.335593 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:36:00.336523 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:36:00.336523 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:00.336523 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:00.336523 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:36:00.336523 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:36:00.336523 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:36:00.336523 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:36:00.336523 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:36:00.336523 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:36:00.336523 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:36:00.336523 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:36:00.336523 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:36:00.336523 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:36:00.336523 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:36:00.336523 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:36:00.336523 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:36:00.336523 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:36:00.337427 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:36:00.337427 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:36:00.337427 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:36:00.337427 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:36:00.337427 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 447/891] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:36:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:36:00.337718 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbd642b-2db3-4a29-b261-7eb63d5a906c None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:00.387208 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da690502-2350-40ca-ae46-61f056dd37ee None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:00.425487 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52669f29-d782-4cb8-9b05-29dee050c763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:00.426217 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52669f29-d782-4cb8-9b05-29dee050c763 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:00.426629 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52669f29-d782-4cb8-9b05-29dee050c763 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:00.426933 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52669f29-d782-4cb8-9b05-29dee050c763 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:00.427610 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52669f29-d782-4cb8-9b05-29dee050c763 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:00.429040 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52669f29-d782-4cb8-9b05-29dee050c763 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:00.442807 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-52669f29-d782-4cb8-9b05-29dee050c763 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 22:36:00.442807 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:00.442807 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:36:00.442807 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:36:00.442807 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:36:00.442807 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:36:00.442807 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:36:00.442807 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:36:00.442807 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 22:36:00.442807 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:36:00.442807 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:36:00.442807 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:36:00.442807 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:00.442807 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:00.442807 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:00.442807 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:00.442807 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:00.444031 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:00.444031 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:00.444031 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:00.444031 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:00.444031 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:00.444031 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:00.444031 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:36:00.444031 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:36:00.444031 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:36:00.444031 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:36:00.444031 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:00.444031 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:00.444031 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:36:00.444031 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:36:00.444031 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:36:00.444031 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:36:00.444879 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:36:00.444879 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:36:00.444879 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:36:00.444879 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:36:00.444879 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:36:00.444879 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:36:00.444879 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:36:00.444879 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:36:00.444879 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:36:00.444879 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:36:00.444879 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:36:00.444879 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:36:00.444879 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 22:36:00.444879 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:36:00.444879 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 445/892] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Jul 21 22:36:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:36:00.445892 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52669f29-d782-4cb8-9b05-29dee050c763 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:00.446061 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbd642b-2db3-4a29-b261-7eb63d5a906c None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:00.450025 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:00.489841 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be5bcc7-39a5-45dd-b247-e88e561e6173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:00.490506 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be5bcc7-39a5-45dd-b247-e88e561e6173 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:00.490808 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be5bcc7-39a5-45dd-b247-e88e561e6173 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:00.491078 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be5bcc7-39a5-45dd-b247-e88e561e6173 None admin] PATH_INFO: `/v3/services` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:00.491798 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be5bcc7-39a5-45dd-b247-e88e561e6173 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:00.492097 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be5bcc7-39a5-45dd-b247-e88e561e6173 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:00.493462 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be5bcc7-39a5-45dd-b247-e88e561e6173 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:00.498488 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 448/893] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 21 22:36:00 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:00.503761 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be5bcc7-39a5-45dd-b247-e88e561e6173 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:00.561769 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52669f29-d782-4cb8-9b05-29dee050c763 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:00.562592 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:00.599785 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb35335-af65-4529-b09f-ad9f06a829a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:00.600468 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb35335-af65-4529-b09f-ad9f06a829a5 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:00.600862 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb35335-af65-4529-b09f-ad9f06a829a5 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:00.601410 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb35335-af65-4529-b09f-ad9f06a829a5 None admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:00.602144 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb35335-af65-4529-b09f-ad9f06a829a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:00.602450 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb35335-af65-4529-b09f-ad9f06a829a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:00.603964 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb35335-af65-4529-b09f-ad9f06a829a5 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:00.608458 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be5bcc7-39a5-45dd-b247-e88e561e6173 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:00.618989 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 446/894] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:36:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:00.620134 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb35335-af65-4529-b09f-ad9f06a829a5 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:00.644324 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25da7da-710a-46e6-aba2-768351322458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:00.645005 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25da7da-710a-46e6-aba2-768351322458 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:00.645873 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25da7da-710a-46e6-aba2-768351322458 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:00.646153 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25da7da-710a-46e6-aba2-768351322458 None admin] PATH_INFO: `/v3/services` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:00.646947 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25da7da-710a-46e6-aba2-768351322458 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:00.647256 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25da7da-710a-46e6-aba2-768351322458 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:00.648631 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25da7da-710a-46e6-aba2-768351322458 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:00.659278 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 449/895] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Jul 21 22:36:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:00.660157 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25da7da-710a-46e6-aba2-768351322458 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:00.690318 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:00.734151 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb35335-af65-4529-b09f-ad9f06a829a5 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:00.771985 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25da7da-710a-46e6-aba2-768351322458 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:00.772462 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b73a4-f0fe-4b13-8d36-9b40c476f71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:00.773149 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b73a4-f0fe-4b13-8d36-9b40c476f71b None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:00.773483 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b73a4-f0fe-4b13-8d36-9b40c476f71b None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:00.773758 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b73a4-f0fe-4b13-8d36-9b40c476f71b None admin] PATH_INFO: `/v3/services` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:00.774492 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169b73a4-f0fe-4b13-8d36-9b40c476f71b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:00.774822 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169b73a4-f0fe-4b13-8d36-9b40c476f71b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:00.776174 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169b73a4-f0fe-4b13-8d36-9b40c476f71b None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:00.785881 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 447/896] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 21 22:36:00 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:00.785881 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b73a4-f0fe-4b13-8d36-9b40c476f71b None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:00.798484 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:00.814218 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a21ae2d-832e-46c1-b577-8965f313b98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:00.815043 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a21ae2d-832e-46c1-b577-8965f313b98f None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:00.815331 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a21ae2d-832e-46c1-b577-8965f313b98f None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:00.824439 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a21ae2d-832e-46c1-b577-8965f313b98f None admin] PATH_INFO: `/v3/projects/service` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:00.824439 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a21ae2d-832e-46c1-b577-8965f313b98f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:00.824613 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a21ae2d-832e-46c1-b577-8965f313b98f None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:00.838860 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-5a21ae2d-832e-46c1-b577-8965f313b98f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:36:00.838860 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:00.838860 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:00.838860 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:00.838860 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:00.838860 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:00.838860 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:00.838860 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:00.838860 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:00.838860 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:00.838860 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:00.838860 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:00.838860 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:36:00.838860 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:36:00.838860 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:36:00.839794 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:36:00.839794 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:00.839794 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:00.839794 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:36:00.839794 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:36:00.839794 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:36:00.839794 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:36:00.839794 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:36:00.839794 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:36:00.839794 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:36:00.839794 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:36:00.839794 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:36:00.839794 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:36:00.839794 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:36:00.839794 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:36:00.839794 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:36:00.839794 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:36:00.840667 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:36:00.840667 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:36:00.840667 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:36:00.840667 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:36:00.840667 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:36:00.840667 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:36:00.840667 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:36:00.840667 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:36:00.840667 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 450/897] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:36:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:36:00.841767 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a21ae2d-832e-46c1-b577-8965f313b98f None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:00.898637 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b73a4-f0fe-4b13-8d36-9b40c476f71b None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:00.926153 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:00.935181 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81634fa9-cc62-4921-81a3-5529c6c3089d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:00.935854 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81634fa9-cc62-4921-81a3-5529c6c3089d None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:00.936144 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81634fa9-cc62-4921-81a3-5529c6c3089d None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:00.936426 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81634fa9-cc62-4921-81a3-5529c6c3089d None admin] PATH_INFO: `/v3/services` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:00.937136 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81634fa9-cc62-4921-81a3-5529c6c3089d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:00.937449 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81634fa9-cc62-4921-81a3-5529c6c3089d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:00.938834 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81634fa9-cc62-4921-81a3-5529c6c3089d None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:00.954890 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 448/898] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Jul 21 22:36:00 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:00.954890 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81634fa9-cc62-4921-81a3-5529c6c3089d None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:00.970155 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a21ae2d-832e-46c1-b577-8965f313b98f None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:01.022394 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef44f33-a434-4bf5-a9a4-2e3bbd816ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:01.023407 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef44f33-a434-4bf5-a9a4-2e3bbd816ec4 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:01.023697 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef44f33-a434-4bf5-a9a4-2e3bbd816ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:01.024719 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef44f33-a434-4bf5-a9a4-2e3bbd816ec4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:01.025720 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef44f33-a434-4bf5-a9a4-2e3bbd816ec4 None admin] RBAC: Adding query filter params (interface=public, service_id=a1134c3f7ed841b59859d4a9b8c4829a, region_id=RegionOne) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:01.026167 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef44f33-a434-4bf5-a9a4-2e3bbd816ec4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:01.027905 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef44f33-a434-4bf5-a9a4-2e3bbd816ec4 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:01.036668 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 451/899] 10.222.0.52 () {56 vars in 1251 bytes} [Thu Jul 21 22:36:00 2022] GET /identity/v3/endpoints?service_id=a1134c3f7ed841b59859d4a9b8c4829a&interface=public®ion_id=RegionOne => generated 194 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:01.041494 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef44f33-a434-4bf5-a9a4-2e3bbd816ec4 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:01.064156 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81634fa9-cc62-4921-81a3-5529c6c3089d None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:01.105876 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78de21ed-1e33-4280-a5aa-8cd9a5f9e53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:01.106660 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de21ed-1e33-4280-a5aa-8cd9a5f9e53b None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:01.107045 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de21ed-1e33-4280-a5aa-8cd9a5f9e53b None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:01.107403 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de21ed-1e33-4280-a5aa-8cd9a5f9e53b None admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:01.108279 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78de21ed-1e33-4280-a5aa-8cd9a5f9e53b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:01.108660 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78de21ed-1e33-4280-a5aa-8cd9a5f9e53b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:01.110194 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78de21ed-1e33-4280-a5aa-8cd9a5f9e53b None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:01.112486 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:01.127151 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 449/900] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 22:36:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:01.133817 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78de21ed-1e33-4280-a5aa-8cd9a5f9e53b None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:01.184055 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef44f33-a434-4bf5-a9a4-2e3bbd816ec4 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:01.208999 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:01.233563 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8bfce-3b26-4717-96a5-c77feaedc12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:01.235013 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8bfce-3b26-4717-96a5-c77feaedc12e None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:01.235810 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8bfce-3b26-4717-96a5-c77feaedc12e None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:01.236304 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8bfce-3b26-4717-96a5-c77feaedc12e None admin] PATH_INFO: `/v3/services` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:01.237346 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d8bfce-3b26-4717-96a5-c77feaedc12e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:01.239642 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d8bfce-3b26-4717-96a5-c77feaedc12e None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:01.255158 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 452/901] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Jul 21 22:36:01 2022] GET /identity/v3/services => generated 1268 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:36:01.264884 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78de21ed-1e33-4280-a5aa-8cd9a5f9e53b None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:01.304570 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac8f74a-d6cb-4520-ab36-c6e943252ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:01.305244 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac8f74a-d6cb-4520-ab36-c6e943252ae7 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:01.305529 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac8f74a-d6cb-4520-ab36-c6e943252ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:01.305808 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac8f74a-d6cb-4520-ab36-c6e943252ae7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:01.306655 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac8f74a-d6cb-4520-ab36-c6e943252ae7 None admin] RBAC: Adding query filter params (role.id=6a667c5557a5498f943c65b692078b57, scope.project.id=aa11ab8229b642ce962193b74001d3bc, user.id=63cc904578554aea84ff1917a5ed075e) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:01.307008 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac8f74a-d6cb-4520-ab36-c6e943252ae7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:01.308474 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac8f74a-d6cb-4520-ab36-c6e943252ae7 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:01.315467 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 450/902] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 22:36:01 2022] GET /identity/v3/role_assignments?user.id=63cc904578554aea84ff1917a5ed075e&scope.project.id=aa11ab8229b642ce962193b74001d3bc&role.id=6a667c5557a5498f943c65b692078b57 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:36:02.029976 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e5beb-2a18-4306-a335-d3be028f2f82 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:02.030495 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e5beb-2a18-4306-a335-d3be028f2f82 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:02.030495 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e5beb-2a18-4306-a335-d3be028f2f82 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:02.032139 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 453/903] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:36:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:36:02.037549 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e4b2b-423c-41ea-8cb5-549c0742bd76 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:02.037948 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e4b2b-423c-41ea-8cb5-549c0742bd76 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:02.038146 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e4b2b-423c-41ea-8cb5-549c0742bd76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:02.054884 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-c45e4b2b-423c-41ea-8cb5-549c0742bd76 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:02.056060 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-c45e4b2b-423c-41ea-8cb5-549c0742bd76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:02.130880 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 451/904] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:36:02 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:36:02.150728 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6e5beb-2a18-4306-a335-d3be028f2f82 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:02.256947 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6e5beb-2a18-4306-a335-d3be028f2f82 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:02.266059 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:02.291534 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66632f4-720e-4784-946a-99c5f6841e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:02.292147 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66632f4-720e-4784-946a-99c5f6841e6e None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:02.292427 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66632f4-720e-4784-946a-99c5f6841e6e None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:02.292701 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66632f4-720e-4784-946a-99c5f6841e6e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:02.293343 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66632f4-720e-4784-946a-99c5f6841e6e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:02.294689 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66632f4-720e-4784-946a-99c5f6841e6e None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:02.306904 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-b66632f4-720e-4784-946a-99c5f6841e6e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 22:36:02.306904 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:02.306904 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:36:02.306904 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:36:02.306904 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:36:02.306904 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:36:02.306904 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:36:02.306904 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:36:02.306904 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 22:36:02.306904 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:36:02.306904 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:36:02.306904 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:36:02.306904 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:02.306904 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:02.306904 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:02.306904 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:02.307818 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:02.307818 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:02.307818 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:02.307818 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:02.307818 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:02.307818 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:02.307818 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:02.307818 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:36:02.307818 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:36:02.307818 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:36:02.307818 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:36:02.307818 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:02.307818 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:02.307818 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:36:02.307818 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:36:02.307818 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:36:02.308641 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:36:02.308641 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:36:02.308641 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:36:02.308641 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:36:02.308641 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:36:02.308641 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:36:02.308641 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:36:02.308641 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:36:02.308641 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:36:02.308641 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:36:02.308641 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:36:02.308641 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:36:02.308641 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:36:02.308641 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 22:36:02.308641 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:36:02.313041 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 454/905] 10.222.0.52 () {56 vars in 1132 bytes} [Thu Jul 21 22:36:02 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:36:02.314438 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45e4b2b-423c-41ea-8cb5-549c0742bd76 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:02.363101 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:02.423275 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45e4b2b-423c-41ea-8cb5-549c0742bd76 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:02.460835 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da7a3f-c0ca-46ba-b337-4816d39301b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:02.461514 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da7a3f-c0ca-46ba-b337-4816d39301b7 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:02.461797 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da7a3f-c0ca-46ba-b337-4816d39301b7 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:02.462070 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da7a3f-c0ca-46ba-b337-4816d39301b7 None admin] PATH_INFO: `/v3/services` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:02.462797 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5da7a3f-c0ca-46ba-b337-4816d39301b7 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:02.463123 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5da7a3f-c0ca-46ba-b337-4816d39301b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:02.464486 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5da7a3f-c0ca-46ba-b337-4816d39301b7 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:02.469591 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 452/906] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Jul 21 22:36:02 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:02.477979 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66632f4-720e-4784-946a-99c5f6841e6e None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:02.584711 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66632f4-720e-4784-946a-99c5f6841e6e None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:02.619355 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8063cb40-1386-4a44-ac22-0339d9f9adfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:02.620026 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8063cb40-1386-4a44-ac22-0339d9f9adfa None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:02.620316 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8063cb40-1386-4a44-ac22-0339d9f9adfa None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:02.620594 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8063cb40-1386-4a44-ac22-0339d9f9adfa None admin] PATH_INFO: `/v3/services` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:02.621302 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8063cb40-1386-4a44-ac22-0339d9f9adfa None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:02.621612 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8063cb40-1386-4a44-ac22-0339d9f9adfa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:02.622998 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8063cb40-1386-4a44-ac22-0339d9f9adfa None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:02.628321 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 455/907] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Jul 21 22:36:02 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:02.635873 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da7a3f-c0ca-46ba-b337-4816d39301b7 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:02.739287 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da7a3f-c0ca-46ba-b337-4816d39301b7 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:02.749988 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:02.774905 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38be8dac-b4c8-4950-9130-59610fc89fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:02.775561 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be8dac-b4c8-4950-9130-59610fc89fb4 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:02.775841 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be8dac-b4c8-4950-9130-59610fc89fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:02.776110 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be8dac-b4c8-4950-9130-59610fc89fb4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:02.776892 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38be8dac-b4c8-4950-9130-59610fc89fb4 None admin] RBAC: Adding query filter params (interface=public, service_id=d1e1a82025e04796a077b3e83464c8e0, region_id=RegionOne) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:02.777197 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38be8dac-b4c8-4950-9130-59610fc89fb4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:02.779232 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38be8dac-b4c8-4950-9130-59610fc89fb4 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:02.789105 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 453/908] 10.222.0.52 () {56 vars in 1251 bytes} [Thu Jul 21 22:36:02 2022] GET /identity/v3/endpoints?service_id=d1e1a82025e04796a077b3e83464c8e0&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:02.790440 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8063cb40-1386-4a44-ac22-0339d9f9adfa None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:02.826377 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:02.893847 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8063cb40-1386-4a44-ac22-0339d9f9adfa None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:02.929943 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c2321f-e388-4811-a6be-4dffb3331cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:02.930620 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c2321f-e388-4811-a6be-4dffb3331cec None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:02.930942 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c2321f-e388-4811-a6be-4dffb3331cec None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:02.931224 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c2321f-e388-4811-a6be-4dffb3331cec None admin] PATH_INFO: `/v3/services` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:02.931911 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c2321f-e388-4811-a6be-4dffb3331cec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:02.933278 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c2321f-e388-4811-a6be-4dffb3331cec None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:02.939481 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 456/909] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Jul 21 22:36:02 2022] GET /identity/v3/services => generated 1268 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:36:03.146124 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b23583-e5fa-4301-93a4-bf787b25a6fc None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:03.147274 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b23583-e5fa-4301-93a4-bf787b25a6fc None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:03.147781 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b23583-e5fa-4301-93a4-bf787b25a6fc None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:03.149652 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 454/910] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:36:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:36:03.156593 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b583b-2b6f-4a7f-be49-6cefe55081e3 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:03.157460 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b583b-2b6f-4a7f-be49-6cefe55081e3 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:03.157958 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b583b-2b6f-4a7f-be49-6cefe55081e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:03.175698 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-d87b583b-2b6f-4a7f-be49-6cefe55081e3 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:03.177757 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-d87b583b-2b6f-4a7f-be49-6cefe55081e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:03.250881 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 457/911] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:36:03 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:36:03.274596 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b23583-e5fa-4301-93a4-bf787b25a6fc None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:03.379287 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b23583-e5fa-4301-93a4-bf787b25a6fc None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:03.405984 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0915389e-d6e9-4e03-a7d8-fa3aef13e944 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:03.406652 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0915389e-d6e9-4e03-a7d8-fa3aef13e944 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:03.407206 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0915389e-d6e9-4e03-a7d8-fa3aef13e944 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:03.409175 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 458/912] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:36:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:36:03.414426 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03b8db-6ec6-41cf-bb5e-492ca1c84093 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:03.414715 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03b8db-6ec6-41cf-bb5e-492ca1c84093 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:03.415019 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03b8db-6ec6-41cf-bb5e-492ca1c84093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:03.421095 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a11a72e-0d90-4b28-bc67-88a6b04b1418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:03.421775 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a11a72e-0d90-4b28-bc67-88a6b04b1418 None admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:03.422061 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a11a72e-0d90-4b28-bc67-88a6b04b1418 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:03.422334 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a11a72e-0d90-4b28-bc67-88a6b04b1418 None admin] PATH_INFO: `/v3/services` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:03.423134 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a11a72e-0d90-4b28-bc67-88a6b04b1418 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:03.424468 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a11a72e-0d90-4b28-bc67-88a6b04b1418 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:03.434971 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-ee03b8db-6ec6-41cf-bb5e-492ca1c84093 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:03.436158 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-ee03b8db-6ec6-41cf-bb5e-492ca1c84093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:03.462666 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:03.512917 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 455/913] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jul 21 22:36:03 2022] POST /identity/v3/services => generated 241 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:36:03.513552 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 459/914] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:36:03 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:36:03.514830 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c8800c-f782-48ad-9064-35825f95d38c None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:03.515544 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c8800c-f782-48ad-9064-35825f95d38c None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:03.516009 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c8800c-f782-48ad-9064-35825f95d38c None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:03.517494 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 456/915] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:36:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:36:03.526752 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197b032-87cd-4769-a443-e060e47a5498 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:03.527382 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197b032-87cd-4769-a443-e060e47a5498 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:03.527839 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197b032-87cd-4769-a443-e060e47a5498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:03.539834 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c8800c-f782-48ad-9064-35825f95d38c None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:03.550761 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-a197b032-87cd-4769-a443-e060e47a5498 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:03.553897 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-a197b032-87cd-4769-a443-e060e47a5498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:03.591412 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:03.670895 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 460/916] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:36:03 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:36:03.674306 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c8800c-f782-48ad-9064-35825f95d38c None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:03.691070 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a197b032-87cd-4769-a443-e060e47a5498 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:03.720745 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9bbc1c-64c7-4bfc-80bb-104a19241bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:03.721541 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9bbc1c-64c7-4bfc-80bb-104a19241bff None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:03.721821 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9bbc1c-64c7-4bfc-80bb-104a19241bff None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:03.722088 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9bbc1c-64c7-4bfc-80bb-104a19241bff None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:03.722731 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9bbc1c-64c7-4bfc-80bb-104a19241bff None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:03.724144 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9bbc1c-64c7-4bfc-80bb-104a19241bff None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:03.738171 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-5d9bbc1c-64c7-4bfc-80bb-104a19241bff None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 22:36:03.738171 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:03.738171 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:36:03.738171 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:36:03.738171 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:36:03.738171 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:36:03.738171 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:36:03.738171 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:36:03.738171 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 22:36:03.738171 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:36:03.738171 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:36:03.738171 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:36:03.738171 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:03.738171 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:03.738171 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:03.738171 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:03.738171 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:03.739825 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:03.739825 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:03.739825 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:03.739825 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:03.739825 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:03.739825 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:03.739825 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:36:03.739825 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:36:03.739825 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:36:03.739825 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:36:03.739825 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:03.739825 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:03.739825 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:36:03.739825 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:36:03.739825 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:36:03.739825 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:36:03.740670 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:36:03.740670 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:36:03.740670 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:36:03.740670 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:36:03.740670 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:36:03.740670 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:36:03.740670 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:36:03.740670 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:36:03.740670 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:36:03.740670 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:36:03.740670 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:36:03.740670 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:36:03.740670 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 22:36:03.740670 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:36:03.740670 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 457/917] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 21 22:36:03 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:36:03.745848 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9bbc1c-64c7-4bfc-80bb-104a19241bff None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:03.794510 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:03.810361 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a197b032-87cd-4769-a443-e060e47a5498 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:03.852897 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47870e6-ceb2-4b16-8941-740af1e6e314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:03.853780 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47870e6-ceb2-4b16-8941-740af1e6e314 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:03.854064 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47870e6-ceb2-4b16-8941-740af1e6e314 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:03.854337 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47870e6-ceb2-4b16-8941-740af1e6e314 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:03.868802 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47870e6-ceb2-4b16-8941-740af1e6e314 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:36:03.877100 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:03.879202 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-d47870e6-ceb2-4b16-8941-740af1e6e314 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 22:36:03.879202 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:03.879202 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:03.879202 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:03.879202 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:03.879202 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:03.879202 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:03.879202 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:03.879202 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:03.879202 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:03.879202 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:03.879202 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:03.879202 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:36:03.879202 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:36:03.879202 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:36:03.879202 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:36:03.880884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:36:03.880884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:36:03.880884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:36:03.880884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:36:03.880884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:03.880884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:03.880884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:36:03.880884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:36:03.880884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:36:03.880884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:36:03.880884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:36:03.880884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:36:03.880884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:36:03.880884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:36:03.880884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:36:03.880884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:36:03.881824 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:36:03.881824 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:36:03.881824 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:36:03.881824 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:36:03.881824 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:36:03.881824 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:36:03.881824 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:36:03.881824 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:36:03.881824 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:36:03.881824 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:36:03.881824 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 22:36:03.881824 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:36:03.881824 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 461/918] 10.222.0.52 () {56 vars in 1120 bytes} [Thu Jul 21 22:36:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:36:03.887165 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47870e6-ceb2-4b16-8941-740af1e6e314 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:03.889357 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9bbc1c-64c7-4bfc-80bb-104a19241bff None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:03.931825 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1868538d-e01b-44e8-8eeb-3946e33917d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:03.932421 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1868538d-e01b-44e8-8eeb-3946e33917d3 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:03.932705 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1868538d-e01b-44e8-8eeb-3946e33917d3 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:03.932987 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1868538d-e01b-44e8-8eeb-3946e33917d3 None admin] PATH_INFO: `/v3/services` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:03.934601 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1868538d-e01b-44e8-8eeb-3946e33917d3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:03.934940 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1868538d-e01b-44e8-8eeb-3946e33917d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:03.936316 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1868538d-e01b-44e8-8eeb-3946e33917d3 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:03.946817 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 458/919] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 21 22:36:03 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:36:03.949642 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1868538d-e01b-44e8-8eeb-3946e33917d3 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:04.009756 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47870e6-ceb2-4b16-8941-740af1e6e314 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:04.046226 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d99c89-ca9f-49bb-a7b9-c0740b820f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:04.046953 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d99c89-ca9f-49bb-a7b9-c0740b820f61 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:04.047240 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d99c89-ca9f-49bb-a7b9-c0740b820f61 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:04.047519 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d99c89-ca9f-49bb-a7b9-c0740b820f61 None admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:04.048270 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d99c89-ca9f-49bb-a7b9-c0740b820f61 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:04.048578 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d99c89-ca9f-49bb-a7b9-c0740b820f61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:04.049949 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d99c89-ca9f-49bb-a7b9-c0740b820f61 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:04.063721 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 462/920] 10.222.0.52 () {56 vars in 1115 bytes} [Thu Jul 21 22:36:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:04.065400 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d99c89-ca9f-49bb-a7b9-c0740b820f61 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:04.080708 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:04.090832 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1868538d-e01b-44e8-8eeb-3946e33917d3 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:04.137318 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655eeeaa-fb46-4edf-b6b3-c181ab9506e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:04.150342 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655eeeaa-fb46-4edf-b6b3-c181ab9506e5 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:04.150342 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655eeeaa-fb46-4edf-b6b3-c181ab9506e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:04.150342 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655eeeaa-fb46-4edf-b6b3-c181ab9506e5 None admin] PATH_INFO: `/v3/services` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:04.150342 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655eeeaa-fb46-4edf-b6b3-c181ab9506e5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:04.150342 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655eeeaa-fb46-4edf-b6b3-c181ab9506e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:04.150342 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655eeeaa-fb46-4edf-b6b3-c181ab9506e5 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:04.152778 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 459/921] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Jul 21 22:36:03 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:04.153929 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655eeeaa-fb46-4edf-b6b3-c181ab9506e5 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:04.185345 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:04.188903 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d99c89-ca9f-49bb-a7b9-c0740b820f61 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:04.226736 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e86694f-7919-4d9d-b3e4-c5178283e2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:04.227935 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e86694f-7919-4d9d-b3e4-c5178283e2c0 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:04.228521 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e86694f-7919-4d9d-b3e4-c5178283e2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:04.229026 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e86694f-7919-4d9d-b3e4-c5178283e2c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:04.240485 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e86694f-7919-4d9d-b3e4-c5178283e2c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:04.242236 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e86694f-7919-4d9d-b3e4-c5178283e2c0 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:04.260574 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-5e86694f-7919-4d9d-b3e4-c5178283e2c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:36:04.260574 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:04.260574 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:04.260574 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:04.260574 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:04.260574 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:04.260574 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:04.260574 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:04.260574 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:04.260574 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:04.260574 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:04.260574 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:04.260574 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:36:04.260574 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:36:04.260574 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:36:04.261996 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:36:04.261996 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:04.261996 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:04.261996 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:36:04.261996 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:36:04.261996 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:36:04.261996 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:36:04.261996 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:36:04.261996 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:36:04.261996 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:36:04.261996 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:36:04.261996 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:36:04.261996 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:36:04.261996 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:36:04.261996 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:36:04.261996 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:36:04.261996 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:36:04.262987 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:36:04.262987 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:36:04.262987 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:36:04.262987 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:36:04.263242 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655eeeaa-fb46-4edf-b6b3-c181ab9506e5 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:04.271499 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 463/922] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:36:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:36:04.275827 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e86694f-7919-4d9d-b3e4-c5178283e2c0 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:04.298413 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3dfd85-e439-4102-96db-8eab405ac62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:04.299265 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3dfd85-e439-4102-96db-8eab405ac62a None admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:04.299618 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3dfd85-e439-4102-96db-8eab405ac62a None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:04.299964 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3dfd85-e439-4102-96db-8eab405ac62a None admin] PATH_INFO: `/v3/endpoints` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:04.300772 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3dfd85-e439-4102-96db-8eab405ac62a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:04.302458 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3dfd85-e439-4102-96db-8eab405ac62a None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:04.338786 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:04.376155 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 460/923] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Jul 21 22:36:04 2022] POST /identity/v3/endpoints => generated 346 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:36:04.390273 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e86694f-7919-4d9d-b3e4-c5178283e2c0 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:04.436715 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6665c65c-8885-4f37-bb79-daeef6932ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:04.437715 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665c65c-8885-4f37-bb79-daeef6932ec0 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:04.438198 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665c65c-8885-4f37-bb79-daeef6932ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:04.438656 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665c65c-8885-4f37-bb79-daeef6932ec0 None admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:04.439593 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6665c65c-8885-4f37-bb79-daeef6932ec0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:04.440144 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6665c65c-8885-4f37-bb79-daeef6932ec0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:04.441756 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6665c65c-8885-4f37-bb79-daeef6932ec0 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:04.456217 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 464/924] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:36:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:04.463008 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3dfd85-e439-4102-96db-8eab405ac62a None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:04.565543 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:04.587770 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3dfd85-e439-4102-96db-8eab405ac62a None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:04.626003 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1b9b8b-d47c-4b80-8bfc-53c993d33842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:04.627228 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b9b8b-d47c-4b80-8bfc-53c993d33842 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:04.627772 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b9b8b-d47c-4b80-8bfc-53c993d33842 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:04.628270 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b9b8b-d47c-4b80-8bfc-53c993d33842 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:04.637038 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1b9b8b-d47c-4b80-8bfc-53c993d33842 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:04.638810 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1b9b8b-d47c-4b80-8bfc-53c993d33842 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:04.654639 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-cc1b9b8b-d47c-4b80-8bfc-53c993d33842 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 22:36:04.654639 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:04.654639 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:04.654639 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:04.654639 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:04.654639 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:04.654639 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:04.654639 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:04.654639 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:04.654639 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:04.654639 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:04.654639 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:04.654639 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:36:04.654639 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:36:04.654639 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:36:04.654639 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:36:04.655667 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:04.655667 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:04.655667 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:36:04.655667 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:36:04.655667 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:36:04.655667 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:36:04.655667 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:36:04.655667 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:36:04.655667 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:36:04.655667 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:36:04.655667 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:36:04.655667 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:36:04.655667 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:36:04.655667 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:36:04.655667 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:36:04.655667 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:36:04.655667 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:36:04.655667 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:36:04.656615 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:36:04.656615 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:36:04.656615 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:36:04.656615 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:36:04.656615 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:36:04.656615 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:36:04.656615 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:36:04.656615 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:36:04.656615 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 22:36:04.656615 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:36:04.659266 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 461/925] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Jul 21 22:36:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:36:04.665893 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6665c65c-8885-4f37-bb79-daeef6932ec0 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:04.778840 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6665c65c-8885-4f37-bb79-daeef6932ec0 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:04.797053 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:04.815626 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7325da7b-72fd-4197-853b-4cc77a475ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:04.821171 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7325da7b-72fd-4197-853b-4cc77a475ab6 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:04.821171 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7325da7b-72fd-4197-853b-4cc77a475ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:04.821171 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7325da7b-72fd-4197-853b-4cc77a475ab6 None admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:04.821171 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7325da7b-72fd-4197-853b-4cc77a475ab6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:04.821171 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7325da7b-72fd-4197-853b-4cc77a475ab6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:04.821171 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7325da7b-72fd-4197-853b-4cc77a475ab6 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:04.838720 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 465/926] 10.222.0.52 () {56 vars in 1149 bytes} [Thu Jul 21 22:36:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:04.844435 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1b9b8b-d47c-4b80-8bfc-53c993d33842 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:04.949327 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1b9b8b-d47c-4b80-8bfc-53c993d33842 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:04.985384 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2051b824-ab9b-4b73-95d8-770b17ab43e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:04.986185 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2051b824-ab9b-4b73-95d8-770b17ab43e5 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:04.986467 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2051b824-ab9b-4b73-95d8-770b17ab43e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:04.986742 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2051b824-ab9b-4b73-95d8-770b17ab43e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:04.998226 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2051b824-ab9b-4b73-95d8-770b17ab43e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:04.999743 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2051b824-ab9b-4b73-95d8-770b17ab43e5 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:05.001466 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:05.021562 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-2051b824-ab9b-4b73-95d8-770b17ab43e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:36:05.021562 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:05.021562 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:05.021562 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:05.021562 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:05.021562 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:05.021562 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:05.021562 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:05.021562 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:05.021562 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:05.021562 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:05.021562 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:05.021562 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:36:05.021562 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:36:05.021562 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:36:05.022494 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:36:05.022494 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:05.022494 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:05.022494 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:36:05.022494 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:36:05.022494 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:36:05.022494 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:36:05.022494 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:36:05.022494 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:36:05.022494 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:36:05.022494 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:36:05.022494 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:36:05.022494 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:36:05.022494 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:36:05.022494 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:36:05.022494 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:36:05.022494 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:36:05.023657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:36:05.023657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:36:05.023657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:36:05.023657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:36:05.025566 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 462/927] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:36:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:36:05.031046 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7325da7b-72fd-4197-853b-4cc77a475ab6 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:05.052628 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbde6fc6-7d91-4463-9ee8-4368a1e3af55 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:05.052923 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbde6fc6-7d91-4463-9ee8-4368a1e3af55 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:05.053207 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbde6fc6-7d91-4463-9ee8-4368a1e3af55 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:05.054901 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 463/928] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:36:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:36:05.059998 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf6b54-322d-4b53-9624-e2172da13fd6 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:05.060291 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf6b54-322d-4b53-9624-e2172da13fd6 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:05.060577 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf6b54-322d-4b53-9624-e2172da13fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:05.080097 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-21cf6b54-322d-4b53-9624-e2172da13fd6 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:05.081585 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-21cf6b54-322d-4b53-9624-e2172da13fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:05.150177 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7325da7b-72fd-4197-853b-4cc77a475ab6 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:05.171076 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 464/929] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:36:05 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:36:05.186879 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf6b54-322d-4b53-9624-e2172da13fd6 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:05.189406 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f66710-af54-4250-9d71-251ff1f60a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:05.190082 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f66710-af54-4250-9d71-251ff1f60a04 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:05.190367 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f66710-af54-4250-9d71-251ff1f60a04 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:05.190643 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f66710-af54-4250-9d71-251ff1f60a04 None admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:05.192752 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f66710-af54-4250-9d71-251ff1f60a04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:05.193225 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f66710-af54-4250-9d71-251ff1f60a04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:05.195326 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f66710-af54-4250-9d71-251ff1f60a04 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:05.208579 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 466/930] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:36:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:05.214152 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f66710-af54-4250-9d71-251ff1f60a04 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:05.255428 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:05.294593 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf6b54-322d-4b53-9624-e2172da13fd6 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:05.319145 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:05.333095 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f66710-af54-4250-9d71-251ff1f60a04 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:05.334335 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7bb3a6-5e9e-400c-9bf3-67f28e46a252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:05.335413 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7bb3a6-5e9e-400c-9bf3-67f28e46a252 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:05.335902 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7bb3a6-5e9e-400c-9bf3-67f28e46a252 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:05.336394 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7bb3a6-5e9e-400c-9bf3-67f28e46a252 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:05.337274 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7bb3a6-5e9e-400c-9bf3-67f28e46a252 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:05.338891 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7bb3a6-5e9e-400c-9bf3-67f28e46a252 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:05.352401 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-4a7bb3a6-5e9e-400c-9bf3-67f28e46a252 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 22:36:05.352401 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:05.352401 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:36:05.352401 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:36:05.352401 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:36:05.352401 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:36:05.352401 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:36:05.352401 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:36:05.352401 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 22:36:05.352401 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:36:05.352401 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:36:05.352401 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:36:05.352401 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:05.352401 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:05.352401 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:05.352401 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:05.353414 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:05.353414 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:05.353414 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:05.353414 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:05.353414 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:05.353414 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:05.353414 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:05.353414 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:36:05.353414 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:36:05.353414 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:36:05.353414 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:36:05.353414 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:05.353414 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:05.353414 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:36:05.353414 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:36:05.353414 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:36:05.354261 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:36:05.354261 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:36:05.354261 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:36:05.354261 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:36:05.354261 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:36:05.354261 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:36:05.354261 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:36:05.354261 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:36:05.354261 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:36:05.354261 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:36:05.354261 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:36:05.354261 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:36:05.354261 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:36:05.354261 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 22:36:05.354261 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:36:05.362236 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 465/931] 10.222.0.52 () {56 vars in 1132 bytes} [Thu Jul 21 22:36:05 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:36:05.363390 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7bb3a6-5e9e-400c-9bf3-67f28e46a252 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:05.373208 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72ca1e7-8c49-41ae-b5f0-5ecf1c5c445c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:05.374648 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ca1e7-8c49-41ae-b5f0-5ecf1c5c445c None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:05.375059 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ca1e7-8c49-41ae-b5f0-5ecf1c5c445c None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:05.375337 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ca1e7-8c49-41ae-b5f0-5ecf1c5c445c None admin] PATH_INFO: `/v3/projects/service` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:05.385348 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72ca1e7-8c49-41ae-b5f0-5ecf1c5c445c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:05.386866 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72ca1e7-8c49-41ae-b5f0-5ecf1c5c445c None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:05.404188 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-d72ca1e7-8c49-41ae-b5f0-5ecf1c5c445c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:36:05.404188 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:05.404188 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:05.404188 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:05.404188 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:05.404188 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:05.404188 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:05.404188 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:05.404188 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:05.404188 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:05.404188 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:05.404188 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:05.404188 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:36:05.404188 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:36:05.404188 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:36:05.405104 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:36:05.405104 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:05.405104 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:05.405104 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:36:05.405104 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:36:05.405104 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:36:05.405104 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:36:05.405104 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:36:05.405104 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:36:05.405104 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:36:05.405104 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:36:05.405104 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:36:05.405104 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:36:05.405104 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:36:05.405104 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:36:05.405104 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:36:05.405104 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:36:05.405998 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:36:05.405998 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:36:05.405998 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:36:05.405998 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:36:05.405998 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:36:05.405998 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:36:05.405998 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:36:05.405998 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:36:05.410314 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 467/932] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:36:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:36:05.411875 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72ca1e7-8c49-41ae-b5f0-5ecf1c5c445c None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:05.473359 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7bb3a6-5e9e-400c-9bf3-67f28e46a252 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:05.519747 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803bf2ec-7298-4bb9-b00a-387578bf3277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:05.520773 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803bf2ec-7298-4bb9-b00a-387578bf3277 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:05.521290 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803bf2ec-7298-4bb9-b00a-387578bf3277 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:05.521785 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803bf2ec-7298-4bb9-b00a-387578bf3277 None admin] PATH_INFO: `/v3/services` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:05.522715 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803bf2ec-7298-4bb9-b00a-387578bf3277 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:05.523270 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803bf2ec-7298-4bb9-b00a-387578bf3277 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:05.524836 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803bf2ec-7298-4bb9-b00a-387578bf3277 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:05.525943 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:05.535226 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72ca1e7-8c49-41ae-b5f0-5ecf1c5c445c None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:05.536147 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 466/933] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Jul 21 22:36:05 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:05.537321 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803bf2ec-7298-4bb9-b00a-387578bf3277 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:05.574133 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfc4962-8053-4d64-ad74-098780d0d7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:05.574721 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfc4962-8053-4d64-ad74-098780d0d7ad None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:05.575048 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfc4962-8053-4d64-ad74-098780d0d7ad None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:05.575326 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfc4962-8053-4d64-ad74-098780d0d7ad None admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:05.576629 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfc4962-8053-4d64-ad74-098780d0d7ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:05.576629 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfc4962-8053-4d64-ad74-098780d0d7ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:05.578201 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfc4962-8053-4d64-ad74-098780d0d7ad None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:05.596305 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:05.596492 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 468/934] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 22:36:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:36:05.598674 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfc4962-8053-4d64-ad74-098780d0d7ad None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:05.647554 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803bf2ec-7298-4bb9-b00a-387578bf3277 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:05.684002 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ada1846-9fb0-4b5e-aee2-8042541b6af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:05.685061 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada1846-9fb0-4b5e-aee2-8042541b6af6 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:05.685554 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada1846-9fb0-4b5e-aee2-8042541b6af6 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:05.686048 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada1846-9fb0-4b5e-aee2-8042541b6af6 None admin] PATH_INFO: `/v3/services` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:05.686994 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ada1846-9fb0-4b5e-aee2-8042541b6af6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:05.687521 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ada1846-9fb0-4b5e-aee2-8042541b6af6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:05.689106 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ada1846-9fb0-4b5e-aee2-8042541b6af6 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:05.694522 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 467/935] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Jul 21 22:36:05 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:05.696143 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd7f7db-03c6-46fa-b1fe-e94e29feee5d None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:05.696672 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd7f7db-03c6-46fa-b1fe-e94e29feee5d None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:05.697151 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd7f7db-03c6-46fa-b1fe-e94e29feee5d None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:05.698874 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 468/936] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:36:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:36:05.704240 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01430ae-986c-4b8e-85dc-34ab34257bd6 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:05.704695 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01430ae-986c-4b8e-85dc-34ab34257bd6 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:05.704975 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01430ae-986c-4b8e-85dc-34ab34257bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:05.712030 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfc4962-8053-4d64-ad74-098780d0d7ad None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:05.724973 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-d01430ae-986c-4b8e-85dc-34ab34257bd6 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:05.726415 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-d01430ae-986c-4b8e-85dc-34ab34257bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:05.751051 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5985037-dc3b-4123-9dd6-07b100a397ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:05.752084 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5985037-dc3b-4123-9dd6-07b100a397ea None admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:05.752363 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5985037-dc3b-4123-9dd6-07b100a397ea None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:05.752635 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5985037-dc3b-4123-9dd6-07b100a397ea None admin] PATH_INFO: `/v3/projects/aa11ab8229b642ce962193b74001d3bc/users/63cc904578554aea84ff1917a5ed075e/roles/6a667c5557a5498f943c65b692078b57` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:05.754907 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5985037-dc3b-4123-9dd6-07b100a397ea None admin] RBAC: Authorizing `identity:create_grant(project_id=aa11ab8229b642ce962193b74001d3bc, user_id=63cc904578554aea84ff1917a5ed075e, role_id=6a667c5557a5498f943c65b692078b57)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:05.756317 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5985037-dc3b-4123-9dd6-07b100a397ea None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:05.793387 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 469/937] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:36:05 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:36:05.794200 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01430ae-986c-4b8e-85dc-34ab34257bd6 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:05.843286 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:05.867557 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 469/938] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 21 22:36:05 2022] PUT /identity/v3/projects/aa11ab8229b642ce962193b74001d3bc/users/63cc904578554aea84ff1917a5ed075e/roles/6a667c5557a5498f943c65b692078b57 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:36:05.868570 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5985037-dc3b-4123-9dd6-07b100a397ea None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:05.889015 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:05.922302 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01430ae-986c-4b8e-85dc-34ab34257bd6 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:05.974797 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defcceb1-670c-4d73-89d4-4017446ebaa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:05.975895 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defcceb1-670c-4d73-89d4-4017446ebaa4 None admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:05.976655 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defcceb1-670c-4d73-89d4-4017446ebaa4 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:05.977180 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defcceb1-670c-4d73-89d4-4017446ebaa4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:05.979416 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defcceb1-670c-4d73-89d4-4017446ebaa4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:05.981175 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defcceb1-670c-4d73-89d4-4017446ebaa4 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:06.001947 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5985037-dc3b-4123-9dd6-07b100a397ea None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:06.041300 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43685299-6410-4170-9646-bfb677e3617e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:06.042141 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43685299-6410-4170-9646-bfb677e3617e None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:06.042449 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43685299-6410-4170-9646-bfb677e3617e None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:06.042724 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43685299-6410-4170-9646-bfb677e3617e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:06.043417 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43685299-6410-4170-9646-bfb677e3617e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:06.044798 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43685299-6410-4170-9646-bfb677e3617e None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:06.057800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-43685299-6410-4170-9646-bfb677e3617e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 22:36:06.057800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:06.057800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:36:06.057800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:36:06.057800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:36:06.057800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:36:06.057800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:36:06.057800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:36:06.057800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 22:36:06.057800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:36:06.057800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:36:06.057800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:36:06.057800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:06.057800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:06.057800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:06.057800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:06.057800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:06.058907 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:06.058907 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:06.058907 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:06.058907 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:06.058907 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:06.058907 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:06.058907 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:36:06.058907 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:36:06.058907 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:36:06.058907 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:36:06.058907 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:06.058907 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:06.058907 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:36:06.058907 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:36:06.058907 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:36:06.058907 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:36:06.059914 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:36:06.059914 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:36:06.059914 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:36:06.059914 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:36:06.059914 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:36:06.059914 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:36:06.059914 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:36:06.059914 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:36:06.059914 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:36:06.059914 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:36:06.059914 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:36:06.059914 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:36:06.059914 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 22:36:06.059914 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:36:06.059914 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 470/939] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Jul 21 22:36:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:36:06.068619 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43685299-6410-4170-9646-bfb677e3617e None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:06.095126 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 470/940] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Jul 21 22:36:05 2022] POST /identity/v3/endpoints => generated 347 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:36:06.163330 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:06.188384 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43685299-6410-4170-9646-bfb677e3617e None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:06.239224 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ae4c19-ab25-4dac-8b6c-654756b47177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:06.239994 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae4c19-ab25-4dac-8b6c-654756b47177 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:06.240233 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae4c19-ab25-4dac-8b6c-654756b47177 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:06.240506 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae4c19-ab25-4dac-8b6c-654756b47177 None admin] PATH_INFO: `/v3/services` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:06.241261 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ae4c19-ab25-4dac-8b6c-654756b47177 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:06.241574 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ae4c19-ab25-4dac-8b6c-654756b47177 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:06.242976 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ae4c19-ab25-4dac-8b6c-654756b47177 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:06.253497 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 471/941] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Jul 21 22:36:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:06.255104 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defcceb1-670c-4d73-89d4-4017446ebaa4 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:06.368713 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defcceb1-670c-4d73-89d4-4017446ebaa4 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:06.370426 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:06.404625 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad128de-da48-4bb5-bb98-2822b463dcbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:06.405316 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad128de-da48-4bb5-bb98-2822b463dcbd None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:06.405806 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad128de-da48-4bb5-bb98-2822b463dcbd None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:06.406268 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad128de-da48-4bb5-bb98-2822b463dcbd None admin] PATH_INFO: `/v3/services` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:06.407383 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad128de-da48-4bb5-bb98-2822b463dcbd None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:06.407827 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad128de-da48-4bb5-bb98-2822b463dcbd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:06.409201 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad128de-da48-4bb5-bb98-2822b463dcbd None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:06.414650 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 471/942] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Jul 21 22:36:06 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:36:06.425725 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ae4c19-ab25-4dac-8b6c-654756b47177 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:06.525566 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ae4c19-ab25-4dac-8b6c-654756b47177 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:06.560167 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd02164-b16c-42a3-bc34-fde7a2fd41f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:06.560855 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd02164-b16c-42a3-bc34-fde7a2fd41f6 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:06.561139 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd02164-b16c-42a3-bc34-fde7a2fd41f6 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:06.561413 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd02164-b16c-42a3-bc34-fde7a2fd41f6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:06.562209 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd02164-b16c-42a3-bc34-fde7a2fd41f6 None admin] RBAC: Adding query filter params (interface=public, service_id=9456d1be44da40978e2b9d47e6234a4d, region_id=RegionOne) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:06.562519 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd02164-b16c-42a3-bc34-fde7a2fd41f6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:06.563913 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd02164-b16c-42a3-bc34-fde7a2fd41f6 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:06.573604 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 472/943] 10.222.0.52 () {56 vars in 1251 bytes} [Thu Jul 21 22:36:06 2022] GET /identity/v3/endpoints?service_id=9456d1be44da40978e2b9d47e6234a4d&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:06.575928 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad128de-da48-4bb5-bb98-2822b463dcbd None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:06.593652 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ed4a4-5ccc-46e3-90ef-2cdb037aa18f None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:06.593941 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ed4a4-5ccc-46e3-90ef-2cdb037aa18f None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:06.594218 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ed4a4-5ccc-46e3-90ef-2cdb037aa18f None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:06.595859 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 473/944] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:36:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:36:06.601526 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf79084f-02db-41e8-bf7e-e377de29fa71 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:06.601814 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf79084f-02db-41e8-bf7e-e377de29fa71 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:06.602096 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf79084f-02db-41e8-bf7e-e377de29fa71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:06.607364 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:06.620279 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-bf79084f-02db-41e8-bf7e-e377de29fa71 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:06.621118 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-bf79084f-02db-41e8-bf7e-e377de29fa71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:06.698186 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 474/945] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:36:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:36:06.709962 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad128de-da48-4bb5-bb98-2822b463dcbd None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:06.721215 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf79084f-02db-41e8-bf7e-e377de29fa71 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:06.761037 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a06021-26cc-48c9-904e-7106f7f8d8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:06.762039 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a06021-26cc-48c9-904e-7106f7f8d8c1 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:06.762538 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a06021-26cc-48c9-904e-7106f7f8d8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:06.763366 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a06021-26cc-48c9-904e-7106f7f8d8c1 None admin] PATH_INFO: `/v3/services` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:06.766260 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a06021-26cc-48c9-904e-7106f7f8d8c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:06.767896 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a06021-26cc-48c9-904e-7106f7f8d8c1 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:06.775891 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 472/946] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Jul 21 22:36:06 2022] GET /identity/v3/services => generated 1497 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:36:06.854109 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf79084f-02db-41e8-bf7e-e377de29fa71 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:06.896565 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:06.899854 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa95ead-4a06-4448-9054-da319fee1e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:06.900534 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa95ead-4a06-4448-9054-da319fee1e7a None admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:06.900816 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa95ead-4a06-4448-9054-da319fee1e7a None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:06.901150 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa95ead-4a06-4448-9054-da319fee1e7a None admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:06.902233 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa95ead-4a06-4448-9054-da319fee1e7a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:06.904034 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa95ead-4a06-4448-9054-da319fee1e7a None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:06.905515 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: WARNING keystone.server.flask.common [None req-efa95ead-4a06-4448-9054-da319fee1e7a None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 21 22:36:06.956521 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.sql.core [None req-efa95ead-4a06-4448-9054-da319fee1e7a None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 21 22:36:06.956521 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 21 22:36:06.956521 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [parameters: {'id': '15d0d6058dc8440eba496611b6ef5277', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 21 22:36:06.956521 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87167) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 21 22:36:06.964473 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-efa95ead-4a06-4448-9054-da319fee1e7a None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 15d0d6058dc8440eba496611b6ef5277.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 15d0d6058dc8440eba496611b6ef5277. Jul 21 22:36:06.964473 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:06.964473 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 22:36:06.964473 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 22:36:06.964473 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 22:36:06.964473 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 22:36:06.964473 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 22:36:06.964473 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 22:36:06.964473 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 22:36:06.964473 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application conn.query(q) Jul 21 22:36:06.964473 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 22:36:06.964473 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 22:36:06.964473 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 22:36:06.965254 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application result.read() Jul 21 22:36:06.965254 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 22:36:06.965254 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 22:36:06.965254 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 22:36:06.965254 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 22:36:06.965254 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 22:36:06.965254 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 22:36:06.965254 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 22:36:06.965254 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 22:36:06.965254 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 21 22:36:06.965254 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:36:06.965254 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 21 22:36:06.965254 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:36:06.965254 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:06.965254 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 21 22:36:06.965254 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 21 22:36:06.965254 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 21 22:36:06.965254 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 21 22:36:06.966249 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 22:36:06.966249 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application next(self.gen) Jul 21 22:36:06.966249 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 21 22:36:06.966249 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application yield resource Jul 21 22:36:06.966249 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 22:36:06.966249 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application next(self.gen) Jul 21 22:36:06.966249 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 21 22:36:06.966249 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self.session.rollback() Jul 21 22:36:06.966249 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 21 22:36:06.966249 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self.force_reraise() Jul 21 22:36:06.966249 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 21 22:36:06.966249 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise self.value Jul 21 22:36:06.966249 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 21 22:36:06.966249 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 21 22:36:06.966249 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 21 22:36:06.966249 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application session.commit() Jul 21 22:36:06.966249 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 21 22:36:06.967801 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 21 22:36:06.967801 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 21 22:36:06.967801 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self._prepare_impl() Jul 21 22:36:06.967801 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 21 22:36:06.967801 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self.session.flush() Jul 21 22:36:06.967801 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 21 22:36:06.967801 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self._flush(objects) Jul 21 22:36:06.967801 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 21 22:36:06.967801 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 21 22:36:06.967801 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 21 22:36:06.967801 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application compat.raise_( Jul 21 22:36:06.967801 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 22:36:06.967801 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception Jul 21 22:36:06.967801 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 21 22:36:06.967801 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application flush_context.execute() Jul 21 22:36:06.967801 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 21 22:36:06.967801 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rec.execute(self) Jul 21 22:36:06.968692 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 21 22:36:06.968692 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 21 22:36:06.968692 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 21 22:36:06.968692 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 21 22:36:06.968692 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 21 22:36:06.968692 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 21 22:36:06.968692 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 21 22:36:06.968692 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 21 22:36:06.968692 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 21 22:36:06.968692 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 21 22:36:06.968692 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 21 22:36:06.968692 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 21 22:36:06.968692 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 21 22:36:06.968692 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 21 22:36:06.968692 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 21 22:36:06.969476 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 21 22:36:06.969476 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 22:36:06.969476 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception Jul 21 22:36:06.969476 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 22:36:06.969476 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 22:36:06.969476 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 22:36:06.969476 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 22:36:06.969476 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 22:36:06.969476 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 22:36:06.969476 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 22:36:06.969476 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application conn.query(q) Jul 21 22:36:06.969476 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 22:36:06.969476 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 22:36:06.969476 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 22:36:06.969476 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application result.read() Jul 21 22:36:06.969476 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 22:36:06.969476 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 22:36:06.970369 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 22:36:06.970369 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 22:36:06.970369 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 22:36:06.970369 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 22:36:06.970369 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 22:36:06.970369 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 22:36:06.970369 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 21 22:36:06.970369 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 21 22:36:06.970369 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [parameters: {'id': '15d0d6058dc8440eba496611b6ef5277', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 21 22:36:06.970369 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 21 22:36:06.970369 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:36:06.970369 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:36:06.970369 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:36:06.970369 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:06.970369 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 21 22:36:06.972670 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 21 22:36:06.972670 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 21 22:36:06.972670 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 21 22:36:06.972670 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 21 22:36:06.972670 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:36:06.972670 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:36:06.972670 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:36:06.972670 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:06.972670 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:06.972670 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:06.972670 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:06.972670 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:06.972670 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:06.972670 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:06.972670 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:06.972670 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:06.972670 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:06.978586 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:06.978586 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 21 22:36:06.978586 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 21 22:36:06.978586 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:06.978586 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:06.978586 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 21 22:36:06.978586 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 21 22:36:06.978586 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 15d0d6058dc8440eba496611b6ef5277. Jul 21 22:36:06.978586 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:36:06.978586 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 475/947] 10.222.0.52 () {60 vars in 1140 bytes} [Thu Jul 21 22:36:06 2022] POST /identity/v3/projects => generated 262 bytes in 245 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:36:06.979985 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a06021-26cc-48c9-904e-7106f7f8d8c1 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:07.006509 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:07.093722 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a06021-26cc-48c9-904e-7106f7f8d8c1 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:07.130103 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f316e920-89b9-42dd-b834-6500d9a695b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:07.130938 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316e920-89b9-42dd-b834-6500d9a695b8 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:07.131218 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316e920-89b9-42dd-b834-6500d9a695b8 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:07.131491 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316e920-89b9-42dd-b834-6500d9a695b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:07.138474 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f316e920-89b9-42dd-b834-6500d9a695b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:07.139934 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f316e920-89b9-42dd-b834-6500d9a695b8 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:07.154758 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-f316e920-89b9-42dd-b834-6500d9a695b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:36:07.154758 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:07.154758 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:07.154758 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:07.154758 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:07.154758 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:07.154758 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:07.154758 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:07.154758 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:07.154758 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:07.154758 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:07.154758 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:07.154758 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:36:07.154758 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:36:07.154758 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:36:07.156747 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:36:07.156747 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:07.156747 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:07.156747 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:36:07.156747 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:36:07.156747 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:36:07.156747 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:36:07.156747 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:36:07.156747 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:36:07.156747 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:36:07.156747 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:36:07.156747 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:36:07.156747 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:36:07.156747 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:36:07.156747 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:36:07.156747 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:36:07.156747 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:36:07.158878 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:36:07.158878 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:36:07.158878 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:36:07.158878 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:36:07.158878 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:36:07.158878 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:36:07.158878 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:36:07.158878 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:36:07.162795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 473/948] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:36:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:36:07.163004 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa95ead-4a06-4448-9054-da319fee1e7a None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:07.267055 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa95ead-4a06-4448-9054-da319fee1e7a None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:07.302575 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd46c1-0206-4abd-b598-88a67a764ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:07.303263 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd46c1-0206-4abd-b598-88a67a764ce6 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:07.303541 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd46c1-0206-4abd-b598-88a67a764ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:07.303814 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd46c1-0206-4abd-b598-88a67a764ce6 None admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:07.304568 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd46c1-0206-4abd-b598-88a67a764ce6 None admin] RBAC: Adding query filter params (name=service) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:07.304871 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd46c1-0206-4abd-b598-88a67a764ce6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:07.306576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd46c1-0206-4abd-b598-88a67a764ce6 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:07.330903 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 476/949] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Jul 21 22:36:07 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:08.016011 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf29c6-c80c-4161-9091-8bb7168dd4a8 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:08.016542 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf29c6-c80c-4161-9091-8bb7168dd4a8 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:08.016542 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf29c6-c80c-4161-9091-8bb7168dd4a8 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:08.018251 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 474/950] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:36:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:36:08.023823 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b38ccc-36a5-4f65-bcc8-7a256ed089da None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:08.024116 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b38ccc-36a5-4f65-bcc8-7a256ed089da None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:08.024397 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b38ccc-36a5-4f65-bcc8-7a256ed089da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:08.042528 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-b2b38ccc-36a5-4f65-bcc8-7a256ed089da None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:08.044112 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-b2b38ccc-36a5-4f65-bcc8-7a256ed089da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:08.088591 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:08.115088 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 477/951] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:36:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:36:08.138303 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbf29c6-c80c-4161-9091-8bb7168dd4a8 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:08.243657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:08.250262 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbf29c6-c80c-4161-9091-8bb7168dd4a8 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:08.286715 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f790e2c4-b116-47bb-bee8-cf3b2c133e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:08.287480 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790e2c4-b116-47bb-bee8-cf3b2c133e49 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:08.287767 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790e2c4-b116-47bb-bee8-cf3b2c133e49 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:08.288053 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790e2c4-b116-47bb-bee8-cf3b2c133e49 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:08.298711 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f790e2c4-b116-47bb-bee8-cf3b2c133e49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:36:08.307440 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-f790e2c4-b116-47bb-bee8-cf3b2c133e49 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 22:36:08.307440 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:08.307440 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:08.307440 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:08.307440 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:08.307440 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:08.307440 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:08.307440 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:08.307440 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:08.307440 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:08.307440 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:08.307440 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:08.307440 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:36:08.307440 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:36:08.307440 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:36:08.307440 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:36:08.308549 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:36:08.308549 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:36:08.308549 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:36:08.308549 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:36:08.308549 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:08.308549 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:08.308549 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:36:08.308549 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:36:08.308549 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:36:08.308549 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:36:08.308549 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:36:08.308549 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:36:08.308549 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:36:08.308549 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:36:08.308549 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:36:08.308549 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:36:08.309430 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:36:08.309430 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:36:08.309430 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:36:08.309430 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:36:08.309430 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:36:08.309430 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:36:08.309430 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:36:08.309430 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:36:08.309430 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:36:08.309430 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:36:08.309430 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 22:36:08.309430 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:36:08.309430 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 475/952] 10.222.0.52 () {56 vars in 1120 bytes} [Thu Jul 21 22:36:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:36:08.314848 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b38ccc-36a5-4f65-bcc8-7a256ed089da None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:08.319272 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ffb3f2-676b-474b-957e-1e46581fe92c None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:08.319787 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ffb3f2-676b-474b-957e-1e46581fe92c None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:08.320284 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ffb3f2-676b-474b-957e-1e46581fe92c None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:08.322037 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 476/953] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:36:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:36:08.327068 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698cf405-ccc6-4fbd-97f1-5bd219f20a8f None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:08.327563 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698cf405-ccc6-4fbd-97f1-5bd219f20a8f None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:08.328023 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698cf405-ccc6-4fbd-97f1-5bd219f20a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:08.344623 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-698cf405-ccc6-4fbd-97f1-5bd219f20a8f None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:08.345853 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-698cf405-ccc6-4fbd-97f1-5bd219f20a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:08.418300 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 477/954] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:36:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:36:08.426789 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b38ccc-36a5-4f65-bcc8-7a256ed089da None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:08.440480 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698cf405-ccc6-4fbd-97f1-5bd219f20a8f None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:08.464688 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c539cd4-2b8b-4f36-ad79-10278cb22ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:08.465366 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c539cd4-2b8b-4f36-ad79-10278cb22ca6 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:08.465650 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c539cd4-2b8b-4f36-ad79-10278cb22ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:08.465928 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c539cd4-2b8b-4f36-ad79-10278cb22ca6 None admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:08.466680 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c539cd4-2b8b-4f36-ad79-10278cb22ca6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:08.467020 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c539cd4-2b8b-4f36-ad79-10278cb22ca6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:08.468435 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c539cd4-2b8b-4f36-ad79-10278cb22ca6 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:08.478373 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 478/955] 10.222.0.52 () {56 vars in 1115 bytes} [Thu Jul 21 22:36:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:08.483935 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c539cd4-2b8b-4f36-ad79-10278cb22ca6 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:08.543872 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:08.546359 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:08.576299 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698cf405-ccc6-4fbd-97f1-5bd219f20a8f None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:08.607732 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c539cd4-2b8b-4f36-ad79-10278cb22ca6 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:08.617158 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0d1683-0a80-4ec2-b2ce-e86d9cc23f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:08.618112 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d1683-0a80-4ec2-b2ce-e86d9cc23f23 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:08.618405 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d1683-0a80-4ec2-b2ce-e86d9cc23f23 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:08.618734 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d1683-0a80-4ec2-b2ce-e86d9cc23f23 None admin] PATH_INFO: `/v3/services/compute` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:08.619421 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0d1683-0a80-4ec2-b2ce-e86d9cc23f23 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:08.620946 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0d1683-0a80-4ec2-b2ce-e86d9cc23f23 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:08.636538 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-8b0d1683-0a80-4ec2-b2ce-e86d9cc23f23 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 22:36:08.636538 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:08.636538 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:36:08.636538 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:36:08.636538 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:36:08.636538 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:36:08.636538 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:36:08.636538 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:36:08.636538 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 22:36:08.636538 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:36:08.636538 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:36:08.636538 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:36:08.636538 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:08.636538 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:08.636538 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:08.636538 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:08.636538 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:08.637839 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:08.637839 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:08.637839 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:08.637839 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:08.637839 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:08.637839 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:08.637839 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:36:08.637839 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:36:08.637839 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:36:08.637839 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:36:08.637839 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:08.637839 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:08.637839 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:36:08.637839 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:36:08.637839 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:36:08.637839 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:36:08.638926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:36:08.638926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:36:08.638926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:36:08.638926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:36:08.638926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:36:08.638926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:36:08.638926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:36:08.638926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:36:08.638926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:36:08.638926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:36:08.638926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:36:08.638926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:36:08.638926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 22:36:08.638926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:36:08.642690 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 478/956] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:36:08 2022] GET /identity/v3/services/compute => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:36:08.643791 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0d1683-0a80-4ec2-b2ce-e86d9cc23f23 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:08.649383 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78844ccb-5054-4595-bfd9-1f9d3a915931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:08.650363 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78844ccb-5054-4595-bfd9-1f9d3a915931 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:08.650904 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78844ccb-5054-4595-bfd9-1f9d3a915931 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:08.651336 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78844ccb-5054-4595-bfd9-1f9d3a915931 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:08.664401 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78844ccb-5054-4595-bfd9-1f9d3a915931 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:08.666054 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78844ccb-5054-4595-bfd9-1f9d3a915931 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:08.684490 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-78844ccb-5054-4595-bfd9-1f9d3a915931 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:36:08.684490 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:08.684490 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:08.684490 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:08.684490 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:08.684490 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:08.684490 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:08.684490 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:08.684490 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:08.684490 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:08.684490 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:08.684490 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:08.684490 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:36:08.684490 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:36:08.684490 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:36:08.685579 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:36:08.685579 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:08.685579 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:08.685579 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:36:08.685579 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:36:08.685579 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:36:08.685579 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:36:08.685579 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:36:08.685579 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:36:08.685579 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:36:08.685579 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:36:08.685579 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:36:08.685579 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:36:08.685579 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:36:08.685579 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:36:08.685579 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:36:08.685579 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:36:08.686541 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:36:08.686541 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:36:08.686541 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:36:08.686541 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:36:08.688384 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 479/957] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:36:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:36:08.698794 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78844ccb-5054-4595-bfd9-1f9d3a915931 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:08.754302 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0d1683-0a80-4ec2-b2ce-e86d9cc23f23 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:08.793259 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470c2637-520d-477a-87d7-30e4d5f5fc3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:08.794078 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c2637-520d-477a-87d7-30e4d5f5fc3c None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:08.794384 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c2637-520d-477a-87d7-30e4d5f5fc3c None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:08.794731 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c2637-520d-477a-87d7-30e4d5f5fc3c None admin] PATH_INFO: `/v3/services` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:08.795527 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470c2637-520d-477a-87d7-30e4d5f5fc3c None admin] RBAC: Adding query filter params (name=compute) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:08.795989 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470c2637-520d-477a-87d7-30e4d5f5fc3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:08.797412 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470c2637-520d-477a-87d7-30e4d5f5fc3c None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:08.806865 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 479/958] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Jul 21 22:36:08 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:08.809119 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470c2637-520d-477a-87d7-30e4d5f5fc3c None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:08.814983 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:08.823040 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:08.826735 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78844ccb-5054-4595-bfd9-1f9d3a915931 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:08.872749 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab623c40-45e9-49e0-8d23-9b237251579c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:08.873413 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab623c40-45e9-49e0-8d23-9b237251579c None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:08.873695 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab623c40-45e9-49e0-8d23-9b237251579c None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:08.873970 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab623c40-45e9-49e0-8d23-9b237251579c None admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:08.875454 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab623c40-45e9-49e0-8d23-9b237251579c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:08.876006 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab623c40-45e9-49e0-8d23-9b237251579c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:08.878465 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab623c40-45e9-49e0-8d23-9b237251579c None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:08.898089 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 480/959] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:36:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:08.899782 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab623c40-45e9-49e0-8d23-9b237251579c None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:08.944009 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470c2637-520d-477a-87d7-30e4d5f5fc3c None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:08.993539 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc4a3d5-bb93-4fc0-a46b-dbabaca40e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:08.994213 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4a3d5-bb93-4fc0-a46b-dbabaca40e71 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:08.994522 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4a3d5-bb93-4fc0-a46b-dbabaca40e71 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:08.994799 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4a3d5-bb93-4fc0-a46b-dbabaca40e71 None admin] PATH_INFO: `/v3/services` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:08.995581 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc4a3d5-bb93-4fc0-a46b-dbabaca40e71 None admin] RBAC: Adding query filter params (type=compute) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:08.995818 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc4a3d5-bb93-4fc0-a46b-dbabaca40e71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:08.997180 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc4a3d5-bb93-4fc0-a46b-dbabaca40e71 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:09.006799 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 480/960] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Jul 21 22:36:08 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:36:09.006799 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda31645-7715-4abf-8757-b2b7029a953c None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:09.006799 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda31645-7715-4abf-8757-b2b7029a953c None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:09.006799 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda31645-7715-4abf-8757-b2b7029a953c None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:09.006799 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 481/961] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:36:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:36:09.010691 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f430b-278a-4072-9728-2cad35052b8a None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:09.011018 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f430b-278a-4072-9728-2cad35052b8a None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:09.011306 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f430b-278a-4072-9728-2cad35052b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:09.023183 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab623c40-45e9-49e0-8d23-9b237251579c None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:09.032321 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-6e4f430b-278a-4072-9728-2cad35052b8a None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:09.033582 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-6e4f430b-278a-4072-9728-2cad35052b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:09.063453 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60aed7e-b031-4784-8f6e-ea5b55762933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:09.064194 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60aed7e-b031-4784-8f6e-ea5b55762933 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:09.064472 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60aed7e-b031-4784-8f6e-ea5b55762933 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:09.064749 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60aed7e-b031-4784-8f6e-ea5b55762933 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:09.074132 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60aed7e-b031-4784-8f6e-ea5b55762933 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:09.075749 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60aed7e-b031-4784-8f6e-ea5b55762933 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:09.092982 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-b60aed7e-b031-4784-8f6e-ea5b55762933 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 22:36:09.092982 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:09.092982 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:09.092982 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:09.092982 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:09.092982 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:09.092982 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:09.092982 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:09.092982 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:09.092982 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:09.092982 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:09.092982 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:09.092982 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:36:09.092982 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:36:09.092982 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:36:09.092982 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:36:09.093993 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:09.093993 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:09.093993 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:36:09.093993 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:36:09.093993 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:36:09.093993 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:36:09.093993 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:36:09.093993 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:36:09.093993 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:36:09.093993 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:36:09.093993 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:36:09.093993 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:36:09.093993 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:36:09.093993 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:36:09.093993 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:36:09.093993 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:36:09.093993 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:36:09.093993 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:36:09.094971 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:36:09.094971 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:36:09.094971 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:36:09.094971 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:36:09.094971 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:36:09.094971 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:36:09.094971 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:36:09.094971 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:36:09.094971 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 22:36:09.094971 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:36:09.094971 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 481/962] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Jul 21 22:36:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:36:09.103735 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60aed7e-b031-4784-8f6e-ea5b55762933 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:09.124552 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:09.138820 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 482/963] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:36:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:36:09.157059 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4f430b-278a-4072-9728-2cad35052b8a None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:09.162270 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:09.246083 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60aed7e-b031-4784-8f6e-ea5b55762933 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:09.287306 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4f430b-278a-4072-9728-2cad35052b8a None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:09.289798 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9785da75-fa00-491e-8b37-d99636c3b14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:09.291013 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9785da75-fa00-491e-8b37-d99636c3b14f None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:09.295154 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9785da75-fa00-491e-8b37-d99636c3b14f None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:09.295418 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9785da75-fa00-491e-8b37-d99636c3b14f None admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:09.296675 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9785da75-fa00-491e-8b37-d99636c3b14f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:09.296991 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9785da75-fa00-491e-8b37-d99636c3b14f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:09.298434 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9785da75-fa00-491e-8b37-d99636c3b14f None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:09.321796 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 482/964] 10.222.0.52 () {56 vars in 1149 bytes} [Thu Jul 21 22:36:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:09.327774 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9785da75-fa00-491e-8b37-d99636c3b14f None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:09.331633 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f916143e-13de-43fc-82bd-04cdbd51c805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:09.333508 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f916143e-13de-43fc-82bd-04cdbd51c805 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:09.333508 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f916143e-13de-43fc-82bd-04cdbd51c805 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:09.333508 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f916143e-13de-43fc-82bd-04cdbd51c805 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:09.334157 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f916143e-13de-43fc-82bd-04cdbd51c805 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:09.335537 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f916143e-13de-43fc-82bd-04cdbd51c805 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:09.347671 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-f916143e-13de-43fc-82bd-04cdbd51c805 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 22:36:09.347671 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:09.347671 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:36:09.347671 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:36:09.347671 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:36:09.347671 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:36:09.347671 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:36:09.347671 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:36:09.347671 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 22:36:09.347671 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:36:09.347671 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:36:09.347671 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:36:09.347671 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:09.347671 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:09.347671 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:09.347671 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:09.347671 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:09.348761 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:09.348761 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:09.348761 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:09.348761 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:09.348761 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:09.348761 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:09.348761 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:36:09.348761 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:36:09.348761 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:36:09.348761 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:36:09.348761 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:09.348761 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:09.348761 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:36:09.348761 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:36:09.348761 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:36:09.348761 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:36:09.349725 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:36:09.349725 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:36:09.349725 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:36:09.349725 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:36:09.349725 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:36:09.349725 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:36:09.349725 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:36:09.349725 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:36:09.349725 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:36:09.349725 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:36:09.349725 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:36:09.349725 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:36:09.349725 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 22:36:09.349725 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:36:09.354076 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 483/965] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Jul 21 22:36:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:36:09.355458 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f916143e-13de-43fc-82bd-04cdbd51c805 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:09.419162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:09.422254 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:09.444966 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9785da75-fa00-491e-8b37-d99636c3b14f None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:09.457860 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f916143e-13de-43fc-82bd-04cdbd51c805 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:09.484666 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eafa848-9dc0-4dec-a633-7509e04df5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:09.488490 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eafa848-9dc0-4dec-a633-7509e04df5c8 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:09.488490 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eafa848-9dc0-4dec-a633-7509e04df5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:09.488490 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eafa848-9dc0-4dec-a633-7509e04df5c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:09.493408 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3316792c-1961-4afe-b497-fde525a8fe2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:09.494144 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3316792c-1961-4afe-b497-fde525a8fe2c None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:09.496463 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3316792c-1961-4afe-b497-fde525a8fe2c None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:09.496545 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3316792c-1961-4afe-b497-fde525a8fe2c None admin] PATH_INFO: `/v3/services` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:09.497261 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3316792c-1961-4afe-b497-fde525a8fe2c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:09.497566 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3316792c-1961-4afe-b497-fde525a8fe2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:09.498991 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3316792c-1961-4afe-b497-fde525a8fe2c None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:09.505003 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 484/966] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Jul 21 22:36:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:09.510720 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52830ed6-0ad3-46ce-b0fd-0b888d09b535 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:09.511042 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52830ed6-0ad3-46ce-b0fd-0b888d09b535 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:09.511320 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52830ed6-0ad3-46ce-b0fd-0b888d09b535 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:09.512852 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 485/967] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:36:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:36:09.514810 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52830ed6-0ad3-46ce-b0fd-0b888d09b535 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:09.571590 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eafa848-9dc0-4dec-a633-7509e04df5c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:09.573239 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eafa848-9dc0-4dec-a633-7509e04df5c8 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:09.624875 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52830ed6-0ad3-46ce-b0fd-0b888d09b535 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:09.647318 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:09.654322 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:09.660095 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b241cc-c27b-4fde-b69d-1481b1505050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:09.660964 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b241cc-c27b-4fde-b69d-1481b1505050 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:09.661260 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b241cc-c27b-4fde-b69d-1481b1505050 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:09.661539 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b241cc-c27b-4fde-b69d-1481b1505050 None admin] PATH_INFO: `/v3/services` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:09.662238 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b241cc-c27b-4fde-b69d-1481b1505050 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:09.662546 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b241cc-c27b-4fde-b69d-1481b1505050 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:09.663941 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b241cc-c27b-4fde-b69d-1481b1505050 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:09.668776 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 486/968] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Jul 21 22:36:09 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:36:09.668776 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ecafd3-dca6-40f7-b6d6-8619c061ba2e None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:09.668776 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ecafd3-dca6-40f7-b6d6-8619c061ba2e None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:09.668776 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ecafd3-dca6-40f7-b6d6-8619c061ba2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:09.688750 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 483/969] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:36:09 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 22:36:09.689874 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eafa848-9dc0-4dec-a633-7509e04df5c8 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:09.693243 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-a6ecafd3-dca6-40f7-b6d6-8619c061ba2e None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:09.694614 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-a6ecafd3-dca6-40f7-b6d6-8619c061ba2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:09.765564 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 487/970] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:36:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:36:09.766879 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ecafd3-dca6-40f7-b6d6-8619c061ba2e None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:09.810271 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eafa848-9dc0-4dec-a633-7509e04df5c8 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:09.848475 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41768fd0-ebf7-41e3-be12-7babcbd23644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:09.849249 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41768fd0-ebf7-41e3-be12-7babcbd23644 None admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:09.849535 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41768fd0-ebf7-41e3-be12-7babcbd23644 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:09.849814 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41768fd0-ebf7-41e3-be12-7babcbd23644 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:09.850558 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41768fd0-ebf7-41e3-be12-7babcbd23644 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:09.852314 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41768fd0-ebf7-41e3-be12-7babcbd23644 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:09.873885 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ecafd3-dca6-40f7-b6d6-8619c061ba2e None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:09.897737 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:09.908217 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb3e6a-36d0-41ea-a895-2b8cee5304dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:09.909002 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb3e6a-36d0-41ea-a895-2b8cee5304dc None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:09.909283 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb3e6a-36d0-41ea-a895-2b8cee5304dc None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:09.909554 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb3e6a-36d0-41ea-a895-2b8cee5304dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:09.919454 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fb3e6a-36d0-41ea-a895-2b8cee5304dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:09.920888 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fb3e6a-36d0-41ea-a895-2b8cee5304dc None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:09.932734 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 484/971] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Jul 21 22:36:09 2022] POST /identity/v3/endpoints => generated 349 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:36:09.934758 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41768fd0-ebf7-41e3-be12-7babcbd23644 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:09.939230 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-e2fb3e6a-36d0-41ea-a895-2b8cee5304dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:36:09.939230 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:09.939230 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:09.939230 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:09.939230 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:09.939230 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:09.939230 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:09.939230 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:09.939230 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:09.939230 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:09.939230 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:09.939230 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:09.939230 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:36:09.939230 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:36:09.939230 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:36:09.940176 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:36:09.940176 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:09.940176 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:09.940176 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:36:09.940176 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:36:09.940176 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:36:09.940176 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:36:09.940176 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:36:09.940176 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:36:09.940176 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:36:09.940176 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:36:09.940176 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:36:09.940176 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:36:09.940176 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:36:09.940176 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:36:09.940176 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:36:09.940176 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:36:09.941075 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:36:09.941075 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:36:09.941075 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:36:09.941075 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:36:09.943441 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 488/972] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:36:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:36:09.946796 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb3e6a-36d0-41ea-a895-2b8cee5304dc None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:09.994835 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:10.089159 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41768fd0-ebf7-41e3-be12-7babcbd23644 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:10.097595 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb3e6a-36d0-41ea-a895-2b8cee5304dc None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:10.138350 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e936619-f788-4cad-9949-c6f0b8a6090b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:10.138989 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e936619-f788-4cad-9949-c6f0b8a6090b None admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:10.139276 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e936619-f788-4cad-9949-c6f0b8a6090b None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:10.139553 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e936619-f788-4cad-9949-c6f0b8a6090b None admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:10.140259 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e936619-f788-4cad-9949-c6f0b8a6090b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:10.141688 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e936619-f788-4cad-9949-c6f0b8a6090b None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:10.148057 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a15a007-a648-40b2-a133-895473b85c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:10.148729 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a15a007-a648-40b2-a133-895473b85c93 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:10.149010 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a15a007-a648-40b2-a133-895473b85c93 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:10.149286 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a15a007-a648-40b2-a133-895473b85c93 None admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:10.150009 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a15a007-a648-40b2-a133-895473b85c93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:10.150315 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a15a007-a648-40b2-a133-895473b85c93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:10.151700 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a15a007-a648-40b2-a133-895473b85c93 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:10.164789 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 489/973] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:36:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:36:10.170572 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a15a007-a648-40b2-a133-895473b85c93 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:10.182710 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.sql.core [None req-4e936619-f788-4cad-9949-c6f0b8a6090b None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 21 22:36:10.182710 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 21 22:36:10.182710 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [parameters: {'user_id': 'b1adfbd6bd7a49c091e82267fc8c3fc0', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 21 22:36:10.182710 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87167) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 21 22:36:10.190312 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-4e936619-f788-4cad-9949-c6f0b8a6090b None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 21 22:36:10.190312 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:10.190312 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 22:36:10.190312 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 22:36:10.190312 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 22:36:10.190312 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 22:36:10.190312 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 22:36:10.190312 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 22:36:10.190312 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 22:36:10.190312 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application conn.query(q) Jul 21 22:36:10.190312 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 22:36:10.190312 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 22:36:10.190312 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 22:36:10.190312 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application result.read() Jul 21 22:36:10.190312 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 22:36:10.193167 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 22:36:10.193167 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 22:36:10.193167 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 22:36:10.193167 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 22:36:10.193167 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 22:36:10.193167 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 22:36:10.193167 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 22:36:10.193167 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 21 22:36:10.193167 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:36:10.193167 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 21 22:36:10.193167 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:36:10.193167 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:10.193167 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 21 22:36:10.193167 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 21 22:36:10.193167 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 21 22:36:10.193167 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 21 22:36:10.193167 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 22:36:10.193167 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application next(self.gen) Jul 21 22:36:10.194140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 21 22:36:10.194140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application yield resource Jul 21 22:36:10.194140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 22:36:10.194140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application next(self.gen) Jul 21 22:36:10.194140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 21 22:36:10.194140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self.session.rollback() Jul 21 22:36:10.194140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 21 22:36:10.194140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self.force_reraise() Jul 21 22:36:10.194140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 21 22:36:10.194140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise self.value Jul 21 22:36:10.194140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 21 22:36:10.194140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 21 22:36:10.194140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 21 22:36:10.194140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application session.commit() Jul 21 22:36:10.194140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 21 22:36:10.194140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 21 22:36:10.194140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 21 22:36:10.195082 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self._prepare_impl() Jul 21 22:36:10.195082 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 21 22:36:10.195082 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self.session.flush() Jul 21 22:36:10.195082 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 21 22:36:10.195082 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self._flush(objects) Jul 21 22:36:10.195082 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 21 22:36:10.195082 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 21 22:36:10.195082 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 21 22:36:10.195082 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application compat.raise_( Jul 21 22:36:10.195082 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 22:36:10.195082 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception Jul 21 22:36:10.195082 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 21 22:36:10.195082 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application flush_context.execute() Jul 21 22:36:10.195082 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 21 22:36:10.195082 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rec.execute(self) Jul 21 22:36:10.195082 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 21 22:36:10.195082 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 21 22:36:10.195974 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 21 22:36:10.195974 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 21 22:36:10.195974 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 21 22:36:10.195974 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 21 22:36:10.195974 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 21 22:36:10.195974 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 21 22:36:10.195974 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 21 22:36:10.195974 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 21 22:36:10.195974 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 21 22:36:10.195974 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 21 22:36:10.195974 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 21 22:36:10.195974 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 21 22:36:10.195974 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 21 22:36:10.195974 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 21 22:36:10.195974 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 22:36:10.195974 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception Jul 21 22:36:10.196830 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 22:36:10.196830 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 22:36:10.196830 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 22:36:10.196830 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 22:36:10.196830 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 22:36:10.196830 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 22:36:10.196830 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 22:36:10.196830 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application conn.query(q) Jul 21 22:36:10.196830 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 22:36:10.196830 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 22:36:10.196830 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 22:36:10.196830 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application result.read() Jul 21 22:36:10.196830 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 22:36:10.196830 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 22:36:10.196830 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 22:36:10.196830 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 22:36:10.197667 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 22:36:10.197667 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 22:36:10.197667 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 22:36:10.197667 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 22:36:10.197667 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 21 22:36:10.197667 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 21 22:36:10.197667 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [parameters: {'user_id': 'b1adfbd6bd7a49c091e82267fc8c3fc0', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 21 22:36:10.197667 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 21 22:36:10.197667 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:36:10.197667 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:36:10.197667 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:36:10.197667 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:10.197667 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:10.197667 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:10.197667 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:10.198466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:10.198466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:10.198466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:10.198466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:10.198466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:10.198466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:10.198466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:10.198466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 21 22:36:10.198466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 21 22:36:10.198466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:10.198466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:10.198466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:36:10.198466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:36:10.198466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:36:10.198466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:36:10.198466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 21 22:36:10.198466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 21 22:36:10.202562 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 21 22:36:10.202562 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 21 22:36:10.202562 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 21 22:36:10.202562 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 21 22:36:10.202562 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 21 22:36:10.202562 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:36:10.202562 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 485/974] 10.222.0.52 () {60 vars in 1131 bytes} [Thu Jul 21 22:36:09 2022] POST /identity/v3/users => generated 160 bytes in 259 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:36:10.202562 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e936619-f788-4cad-9949-c6f0b8a6090b None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:10.223180 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:10.297716 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:10.308676 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a15a007-a648-40b2-a133-895473b85c93 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:10.317020 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e936619-f788-4cad-9949-c6f0b8a6090b None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:10.356717 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca5e9f4-1e17-41af-94ca-4c97f6ed961c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:10.357341 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95f15c0-f79f-4202-9068-d9f62df5fb13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:10.357815 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f15c0-f79f-4202-9068-d9f62df5fb13 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:10.358168 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f15c0-f79f-4202-9068-d9f62df5fb13 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:10.358512 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f15c0-f79f-4202-9068-d9f62df5fb13 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:10.358983 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5e9f4-1e17-41af-94ca-4c97f6ed961c None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:10.359585 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5e9f4-1e17-41af-94ca-4c97f6ed961c None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:10.360098 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5e9f4-1e17-41af-94ca-4c97f6ed961c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:10.366912 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95f15c0-f79f-4202-9068-d9f62df5fb13 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:10.368762 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95f15c0-f79f-4202-9068-d9f62df5fb13 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:10.369570 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca5e9f4-1e17-41af-94ca-4c97f6ed961c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:10.371393 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca5e9f4-1e17-41af-94ca-4c97f6ed961c None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:10.385129 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-d95f15c0-f79f-4202-9068-d9f62df5fb13 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:36:10.385129 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:10.385129 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:10.385129 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:10.385129 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:10.385129 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:10.385129 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:10.385129 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:10.385129 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:10.385129 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:10.385129 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:10.385129 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:10.385129 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:36:10.385129 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:36:10.385129 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:36:10.386029 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:36:10.386029 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:10.386029 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:10.386029 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:36:10.386029 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:36:10.386029 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:36:10.386029 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:36:10.386029 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:36:10.386029 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:36:10.386029 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:36:10.386029 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:36:10.386029 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:36:10.386029 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:36:10.386029 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:36:10.386029 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:36:10.386029 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:36:10.386029 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:36:10.387323 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:36:10.387323 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:36:10.387323 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:36:10.387323 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:36:10.387323 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:36:10.387323 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:36:10.387323 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:36:10.387323 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:36:10.390594 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-aca5e9f4-1e17-41af-94ca-4c97f6ed961c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 22:36:10.390594 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:10.390594 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:10.390594 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:10.390594 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:10.390594 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:10.390594 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:10.390594 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:10.390594 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:10.390594 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:10.390594 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:10.390594 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:10.390594 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:36:10.390594 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:36:10.390594 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:36:10.390594 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:36:10.391550 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:10.391550 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:10.391550 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:36:10.391550 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:36:10.391550 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:36:10.391550 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:36:10.391550 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:36:10.391550 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:36:10.391550 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:36:10.391550 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:36:10.391550 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:36:10.391550 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:36:10.391550 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:36:10.391550 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:36:10.391550 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:36:10.391550 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:36:10.391550 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:36:10.391550 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:36:10.392487 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:36:10.392487 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:36:10.392487 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:36:10.392487 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:36:10.392487 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:36:10.392487 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:36:10.392487 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:36:10.392487 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:36:10.392487 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 22:36:10.392487 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:36:10.393040 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 490/975] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:36:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:36:10.393040 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95f15c0-f79f-4202-9068-d9f62df5fb13 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:10.400415 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 486/976] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:36:10 2022] GET /identity/v3/users/cinder => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:36:10.401504 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca5e9f4-1e17-41af-94ca-4c97f6ed961c None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:10.501828 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:10.508942 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95f15c0-f79f-4202-9068-d9f62df5fb13 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:10.516632 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca5e9f4-1e17-41af-94ca-4c97f6ed961c None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:10.545763 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6848384f-d99f-4333-9117-2f32961692e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:10.546430 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6848384f-d99f-4333-9117-2f32961692e6 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:10.546715 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6848384f-d99f-4333-9117-2f32961692e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:10.547030 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6848384f-d99f-4333-9117-2f32961692e6 None admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:10.547898 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6848384f-d99f-4333-9117-2f32961692e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:10.548124 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6848384f-d99f-4333-9117-2f32961692e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:10.549552 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6848384f-d99f-4333-9117-2f32961692e6 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:10.556784 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07097d6a-af57-470a-a635-32ffc92aac69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:10.557831 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07097d6a-af57-470a-a635-32ffc92aac69 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:10.558314 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07097d6a-af57-470a-a635-32ffc92aac69 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:10.558915 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07097d6a-af57-470a-a635-32ffc92aac69 None admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:10.559924 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07097d6a-af57-470a-a635-32ffc92aac69 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:10.560450 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07097d6a-af57-470a-a635-32ffc92aac69 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:10.562138 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07097d6a-af57-470a-a635-32ffc92aac69 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:10.567318 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 491/977] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 22:36:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:36:10.568542 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6848384f-d99f-4333-9117-2f32961692e6 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:10.610966 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 487/978] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Jul 21 22:36:10 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:10.680736 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6848384f-d99f-4333-9117-2f32961692e6 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:10.718663 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f62d93-71a4-45d9-a634-93cbe5c2dca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:10.719366 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f62d93-71a4-45d9-a634-93cbe5c2dca3 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:10.719653 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f62d93-71a4-45d9-a634-93cbe5c2dca3 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:10.719931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f62d93-71a4-45d9-a634-93cbe5c2dca3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:10.720772 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f62d93-71a4-45d9-a634-93cbe5c2dca3 None admin] RBAC: Adding query filter params (role.id=6a667c5557a5498f943c65b692078b57, scope.project.id=aa11ab8229b642ce962193b74001d3bc, user.id=63cc904578554aea84ff1917a5ed075e) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:10.721086 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f62d93-71a4-45d9-a634-93cbe5c2dca3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:10.722537 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f62d93-71a4-45d9-a634-93cbe5c2dca3 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:10.734908 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 492/979] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 22:36:10 2022] GET /identity/v3/role_assignments?user.id=63cc904578554aea84ff1917a5ed075e&scope.project.id=aa11ab8229b642ce962193b74001d3bc&role.id=6a667c5557a5498f943c65b692078b57 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:36:11.156095 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bde120-8323-4b59-850f-209ad6f62bae None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:11.156499 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bde120-8323-4b59-850f-209ad6f62bae None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:11.156675 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bde120-8323-4b59-850f-209ad6f62bae None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:11.158138 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 488/980] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:36:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:36:11.163307 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2625d127-d860-4bd2-b93a-45fe4e569701 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:11.163743 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2625d127-d860-4bd2-b93a-45fe4e569701 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:11.163921 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2625d127-d860-4bd2-b93a-45fe4e569701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:11.181095 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-2625d127-d860-4bd2-b93a-45fe4e569701 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:11.182291 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-2625d127-d860-4bd2-b93a-45fe4e569701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:11.198027 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:11.253715 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 493/981] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:36:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:36:11.276344 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bde120-8323-4b59-850f-209ad6f62bae None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:11.279323 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:11.379696 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bde120-8323-4b59-850f-209ad6f62bae None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:11.414061 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48190980-b55b-4ef5-80af-2b9b7d2cea69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:11.414658 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48190980-b55b-4ef5-80af-2b9b7d2cea69 None admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:11.414970 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48190980-b55b-4ef5-80af-2b9b7d2cea69 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:11.415244 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48190980-b55b-4ef5-80af-2b9b7d2cea69 None admin] PATH_INFO: `/v3/services` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:11.415982 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48190980-b55b-4ef5-80af-2b9b7d2cea69 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:11.417334 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48190980-b55b-4ef5-80af-2b9b7d2cea69 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:11.471122 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 489/982] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jul 21 22:36:11 2022] POST /identity/v3/services => generated 242 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:36:12.070705 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91652c27-6427-441b-8d17-d9d2f6706375 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:12.071123 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91652c27-6427-441b-8d17-d9d2f6706375 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:12.071303 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91652c27-6427-441b-8d17-d9d2f6706375 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:12.072669 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 494/983] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:36:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:36:12.078006 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b922b-c6bd-41b5-8625-39eddf514d0c None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:12.078305 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b922b-c6bd-41b5-8625-39eddf514d0c None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:12.078587 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b922b-c6bd-41b5-8625-39eddf514d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:12.095574 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-443b922b-c6bd-41b5-8625-39eddf514d0c None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:12.097094 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-443b922b-c6bd-41b5-8625-39eddf514d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:12.155204 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:12.172668 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 490/984] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:36:12 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:36:12.196731 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91652c27-6427-441b-8d17-d9d2f6706375 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:12.303379 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91652c27-6427-441b-8d17-d9d2f6706375 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:12.336975 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df165f03-04aa-4a14-9f49-03005d82480b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:12.337776 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df165f03-04aa-4a14-9f49-03005d82480b None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:12.338058 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df165f03-04aa-4a14-9f49-03005d82480b None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:12.338326 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df165f03-04aa-4a14-9f49-03005d82480b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:12.338999 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df165f03-04aa-4a14-9f49-03005d82480b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:12.340355 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df165f03-04aa-4a14-9f49-03005d82480b None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:12.353046 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-df165f03-04aa-4a14-9f49-03005d82480b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 22:36:12.353046 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:12.353046 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:36:12.353046 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:36:12.353046 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:36:12.353046 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:36:12.353046 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:36:12.353046 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:36:12.353046 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 22:36:12.353046 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:36:12.353046 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:36:12.353046 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:36:12.353046 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:12.353046 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:12.353046 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:12.353046 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:12.353046 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:12.354006 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:12.354006 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:12.354006 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:12.354006 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:12.354006 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:12.354006 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:12.354006 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:36:12.354006 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:36:12.354006 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:36:12.354006 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:36:12.354006 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:12.354006 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:12.354006 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:36:12.354006 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:36:12.354006 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:36:12.354006 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:36:12.354874 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:36:12.354874 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:36:12.354874 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:36:12.354874 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:36:12.354874 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:36:12.354874 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:36:12.354874 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:36:12.354874 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:36:12.354874 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:36:12.354874 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:36:12.354874 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:36:12.354874 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:36:12.354874 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 22:36:12.354874 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:36:12.354874 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 495/985] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Jul 21 22:36:12 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:36:12.360128 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443b922b-c6bd-41b5-8625-39eddf514d0c None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:12.461517 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443b922b-c6bd-41b5-8625-39eddf514d0c None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:12.503576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3680dd5a-5f72-4b98-be06-4cd11fead79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:12.504757 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680dd5a-5f72-4b98-be06-4cd11fead79f None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:12.505041 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680dd5a-5f72-4b98-be06-4cd11fead79f None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:12.505318 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680dd5a-5f72-4b98-be06-4cd11fead79f None admin] PATH_INFO: `/v3/services` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:12.506033 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3680dd5a-5f72-4b98-be06-4cd11fead79f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:12.506339 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3680dd5a-5f72-4b98-be06-4cd11fead79f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:12.507736 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3680dd5a-5f72-4b98-be06-4cd11fead79f None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:12.517524 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 491/986] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Jul 21 22:36:12 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:12.518651 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df165f03-04aa-4a14-9f49-03005d82480b None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:12.536132 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:12.625348 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df165f03-04aa-4a14-9f49-03005d82480b None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:12.661119 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ad92a4-de52-4ef1-84a5-6544e6594edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:12.661785 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad92a4-de52-4ef1-84a5-6544e6594edd None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:12.662071 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad92a4-de52-4ef1-84a5-6544e6594edd None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:12.662348 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad92a4-de52-4ef1-84a5-6544e6594edd None admin] PATH_INFO: `/v3/services` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:12.663089 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ad92a4-de52-4ef1-84a5-6544e6594edd None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:12.663400 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ad92a4-de52-4ef1-84a5-6544e6594edd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:12.664764 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ad92a4-de52-4ef1-84a5-6544e6594edd None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:12.674440 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 496/987] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Jul 21 22:36:12 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:12.676389 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3680dd5a-5f72-4b98-be06-4cd11fead79f None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:12.732050 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:12.781722 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3680dd5a-5f72-4b98-be06-4cd11fead79f None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:12.808385 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14cd8b0-d1bc-4862-8744-35b26fb0580a None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:12.808677 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14cd8b0-d1bc-4862-8744-35b26fb0580a None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:12.808961 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14cd8b0-d1bc-4862-8744-35b26fb0580a None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:12.810311 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 497/988] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:36:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:36:12.816284 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a697a-7430-4560-a457-f8dd9aad3c14 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:12.817099 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a697a-7430-4560-a457-f8dd9aad3c14 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:12.817378 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a697a-7430-4560-a457-f8dd9aad3c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:12.819135 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dcb0fd-cafa-46bb-94a2-75e7d85b0848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:12.820054 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dcb0fd-cafa-46bb-94a2-75e7d85b0848 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:12.820557 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dcb0fd-cafa-46bb-94a2-75e7d85b0848 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:12.821034 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dcb0fd-cafa-46bb-94a2-75e7d85b0848 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:12.822033 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dcb0fd-cafa-46bb-94a2-75e7d85b0848 None admin] RBAC: Adding query filter params (interface=admin, service_id=9456d1be44da40978e2b9d47e6234a4d, region_id=RegionOne) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:12.822491 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dcb0fd-cafa-46bb-94a2-75e7d85b0848 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:12.823888 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dcb0fd-cafa-46bb-94a2-75e7d85b0848 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:12.829338 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 492/989] 10.222.0.52 () {56 vars in 1249 bytes} [Thu Jul 21 22:36:12 2022] GET /identity/v3/endpoints?service_id=9456d1be44da40978e2b9d47e6234a4d&interface=admin®ion_id=RegionOne => generated 193 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:12.834556 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dcb0fd-cafa-46bb-94a2-75e7d85b0848 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:12.836056 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-3a8a697a-7430-4560-a457-f8dd9aad3c14 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:12.837250 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-3a8a697a-7430-4560-a457-f8dd9aad3c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:12.910109 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 498/990] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:36:12 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:36:12.932306 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8a697a-7430-4560-a457-f8dd9aad3c14 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:12.942634 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:12.947762 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dcb0fd-cafa-46bb-94a2-75e7d85b0848 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:12.964237 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:12.983060 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f81c7-d2e4-45a8-a2ab-eafb21558266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:12.983725 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f81c7-d2e4-45a8-a2ab-eafb21558266 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:12.984027 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f81c7-d2e4-45a8-a2ab-eafb21558266 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:12.984345 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f81c7-d2e4-45a8-a2ab-eafb21558266 None admin] PATH_INFO: `/v3/services` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:12.985030 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5f81c7-d2e4-45a8-a2ab-eafb21558266 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:12.986388 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5f81c7-d2e4-45a8-a2ab-eafb21558266 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:12.993982 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 493/991] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Jul 21 22:36:12 2022] GET /identity/v3/services => generated 1727 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:36:13.042021 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8a697a-7430-4560-a457-f8dd9aad3c14 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:13.079803 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63089e90-4be8-4c5f-8c75-a557d26e5f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:13.080610 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63089e90-4be8-4c5f-8c75-a557d26e5f4b None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:13.080892 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63089e90-4be8-4c5f-8c75-a557d26e5f4b None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:13.081168 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63089e90-4be8-4c5f-8c75-a557d26e5f4b None admin] PATH_INFO: `/v3/services/image` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:13.081895 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63089e90-4be8-4c5f-8c75-a557d26e5f4b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:13.083329 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63089e90-4be8-4c5f-8c75-a557d26e5f4b None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:13.095852 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-63089e90-4be8-4c5f-8c75-a557d26e5f4b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 22:36:13.095852 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:13.095852 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:36:13.095852 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:36:13.095852 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:36:13.095852 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:36:13.095852 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:36:13.095852 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:36:13.095852 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 22:36:13.095852 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:36:13.095852 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:36:13.095852 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:36:13.095852 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:13.095852 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:13.095852 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:13.095852 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:13.095852 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:13.096963 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:13.096963 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:13.096963 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:13.096963 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:13.096963 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:13.096963 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:13.096963 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:36:13.096963 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:36:13.096963 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:36:13.096963 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:36:13.096963 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:13.096963 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:13.096963 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:36:13.096963 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:36:13.096963 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:36:13.096963 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:36:13.098347 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:36:13.098347 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:36:13.098347 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:36:13.098347 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:36:13.098347 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:36:13.098347 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:36:13.098347 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:36:13.098347 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:36:13.098347 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:36:13.098347 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:36:13.098347 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:36:13.098347 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:36:13.098347 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 22:36:13.098347 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:36:13.098347 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 499/992] 10.222.0.52 () {56 vars in 1105 bytes} [Thu Jul 21 22:36:12 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:36:13.104098 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f81c7-d2e4-45a8-a2ab-eafb21558266 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:13.212746 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f81c7-d2e4-45a8-a2ab-eafb21558266 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:13.247173 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5fe80d-9529-4b5e-a010-00addaea9833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:13.247841 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5fe80d-9529-4b5e-a010-00addaea9833 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:13.248123 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5fe80d-9529-4b5e-a010-00addaea9833 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:13.248403 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5fe80d-9529-4b5e-a010-00addaea9833 None admin] PATH_INFO: `/v3/services` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:13.249104 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5fe80d-9529-4b5e-a010-00addaea9833 None admin] RBAC: Adding query filter params (name=image) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:13.249411 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5fe80d-9529-4b5e-a010-00addaea9833 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:13.250786 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5fe80d-9529-4b5e-a010-00addaea9833 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:13.256527 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 494/993] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:36:13 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:13.262363 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63089e90-4be8-4c5f-8c75-a557d26e5f4b None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:13.338366 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:13.367647 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63089e90-4be8-4c5f-8c75-a557d26e5f4b None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:13.402478 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e606a-458d-43f7-9f5d-d984d61e1469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:13.403194 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e606a-458d-43f7-9f5d-d984d61e1469 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:13.403482 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e606a-458d-43f7-9f5d-d984d61e1469 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:13.403757 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e606a-458d-43f7-9f5d-d984d61e1469 None admin] PATH_INFO: `/v3/services` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:13.404466 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e606a-458d-43f7-9f5d-d984d61e1469 None admin] RBAC: Adding query filter params (type=image) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:13.404775 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e606a-458d-43f7-9f5d-d984d61e1469 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:13.406140 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e606a-458d-43f7-9f5d-d984d61e1469 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:13.411012 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 500/994] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:36:13 2022] GET /identity/v3/services?type=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:13.545654 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca47be-1d4a-4253-989b-4970638e07eb None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:13.546171 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca47be-1d4a-4253-989b-4970638e07eb None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:13.546171 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca47be-1d4a-4253-989b-4970638e07eb None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:13.547790 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 495/995] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:36:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:36:13.553645 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42fbea7-cba7-4e25-b3de-dad24f7a174d None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:13.553938 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42fbea7-cba7-4e25-b3de-dad24f7a174d None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:13.554214 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42fbea7-cba7-4e25-b3de-dad24f7a174d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:13.572324 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-d42fbea7-cba7-4e25-b3de-dad24f7a174d None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:13.573600 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-d42fbea7-cba7-4e25-b3de-dad24f7a174d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:13.644117 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 501/996] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:36:13 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:36:13.665996 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca47be-1d4a-4253-989b-4970638e07eb None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:13.715624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:13.773342 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca47be-1d4a-4253-989b-4970638e07eb None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:13.808444 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20af6e8-e3aa-4dbf-ad3f-92fc000a151a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:13.809238 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20af6e8-e3aa-4dbf-ad3f-92fc000a151a None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:13.809523 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20af6e8-e3aa-4dbf-ad3f-92fc000a151a None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:13.809799 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20af6e8-e3aa-4dbf-ad3f-92fc000a151a None admin] PATH_INFO: `/v3/services/compute` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:13.810447 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20af6e8-e3aa-4dbf-ad3f-92fc000a151a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:13.811839 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20af6e8-e3aa-4dbf-ad3f-92fc000a151a None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:13.824239 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-c20af6e8-e3aa-4dbf-ad3f-92fc000a151a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 22:36:13.824239 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:13.824239 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:36:13.824239 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:36:13.824239 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:36:13.824239 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:36:13.824239 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:36:13.824239 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:36:13.824239 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 22:36:13.824239 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:36:13.824239 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:36:13.824239 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:36:13.824239 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:13.824239 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:13.824239 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:13.824239 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:13.824239 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:13.825224 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:13.825224 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:13.825224 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:13.825224 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:13.825224 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:13.825224 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:13.825224 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:36:13.825224 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:36:13.825224 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:36:13.825224 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:36:13.825224 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:13.825224 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:13.825224 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:36:13.825224 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:36:13.825224 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:36:13.825224 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:36:13.826064 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:36:13.826064 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:36:13.826064 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:36:13.826064 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:36:13.826064 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:36:13.826064 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:36:13.826064 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:36:13.826064 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:36:13.826064 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:36:13.826064 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:36:13.826064 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:36:13.826064 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:36:13.826064 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 22:36:13.826064 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:36:13.830928 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 496/997] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:36:13 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:36:13.832558 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42fbea7-cba7-4e25-b3de-dad24f7a174d None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:13.897937 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:13.933954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42fbea7-cba7-4e25-b3de-dad24f7a174d None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:13.967512 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39968465-fe9f-44d0-b343-410213cc0c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:13.968181 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39968465-fe9f-44d0-b343-410213cc0c3a None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:13.968467 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39968465-fe9f-44d0-b343-410213cc0c3a None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:13.968742 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39968465-fe9f-44d0-b343-410213cc0c3a None admin] PATH_INFO: `/v3/services` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:13.969446 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39968465-fe9f-44d0-b343-410213cc0c3a None admin] RBAC: Adding query filter params (name=compute) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:13.969751 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39968465-fe9f-44d0-b343-410213cc0c3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:13.971169 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39968465-fe9f-44d0-b343-410213cc0c3a None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:13.976952 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 502/998] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Jul 21 22:36:13 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:13.982395 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20af6e8-e3aa-4dbf-ad3f-92fc000a151a None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:14.083955 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20af6e8-e3aa-4dbf-ad3f-92fc000a151a None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:14.092967 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:14.119312 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a310c397-2bb3-43af-84e0-d5f4c9f4cc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:14.120009 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310c397-2bb3-43af-84e0-d5f4c9f4cc68 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:14.120297 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310c397-2bb3-43af-84e0-d5f4c9f4cc68 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:14.120576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310c397-2bb3-43af-84e0-d5f4c9f4cc68 None admin] PATH_INFO: `/v3/services` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:14.121314 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a310c397-2bb3-43af-84e0-d5f4c9f4cc68 None admin] RBAC: Adding query filter params (type=compute) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:14.121626 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a310c397-2bb3-43af-84e0-d5f4c9f4cc68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:14.123076 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a310c397-2bb3-43af-84e0-d5f4c9f4cc68 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:14.133899 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 497/999] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Jul 21 22:36:13 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:14.135142 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39968465-fe9f-44d0-b343-410213cc0c3a None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:14.235665 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39968465-fe9f-44d0-b343-410213cc0c3a None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:14.264597 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:14.271659 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4360b0a8-4bb8-4884-9cfa-38b020591081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:14.272374 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360b0a8-4bb8-4884-9cfa-38b020591081 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:14.272660 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360b0a8-4bb8-4884-9cfa-38b020591081 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:14.272986 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360b0a8-4bb8-4884-9cfa-38b020591081 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:14.273837 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4360b0a8-4bb8-4884-9cfa-38b020591081 None admin] RBAC: Adding query filter params (interface=public, service_id=af79774a4d034456aea913f848b5b3bf, region_id=RegionOne) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:14.274153 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4360b0a8-4bb8-4884-9cfa-38b020591081 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:14.275541 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4360b0a8-4bb8-4884-9cfa-38b020591081 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:14.285548 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 503/1000] 10.222.0.52 () {56 vars in 1251 bytes} [Thu Jul 21 22:36:14 2022] GET /identity/v3/endpoints?service_id=af79774a4d034456aea913f848b5b3bf&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:14.286928 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a310c397-2bb3-43af-84e0-d5f4c9f4cc68 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:14.392173 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a310c397-2bb3-43af-84e0-d5f4c9f4cc68 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:14.425971 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5355c837-9c5f-4c01-b4a6-56561c50852d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:14.426633 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355c837-9c5f-4c01-b4a6-56561c50852d None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:14.426949 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355c837-9c5f-4c01-b4a6-56561c50852d None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:14.427230 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355c837-9c5f-4c01-b4a6-56561c50852d None admin] PATH_INFO: `/v3/services` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:14.427907 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5355c837-9c5f-4c01-b4a6-56561c50852d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:14.429258 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5355c837-9c5f-4c01-b4a6-56561c50852d None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:14.435705 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 498/1001] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Jul 21 22:36:14 2022] GET /identity/v3/services => generated 1727 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:36:15.066244 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea202411-0a48-4098-b8f4-281e1ed8b0ca None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:15.066646 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea202411-0a48-4098-b8f4-281e1ed8b0ca None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:15.066849 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea202411-0a48-4098-b8f4-281e1ed8b0ca None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:15.068340 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 504/1002] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:36:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:36:15.073381 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689e5bb-d001-4b20-848c-180e0ac25554 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:15.073833 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689e5bb-d001-4b20-848c-180e0ac25554 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:15.073999 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689e5bb-d001-4b20-848c-180e0ac25554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:15.091507 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-3689e5bb-d001-4b20-848c-180e0ac25554 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:15.092737 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-3689e5bb-d001-4b20-848c-180e0ac25554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:15.122353 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:15.162114 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 499/1003] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:36:15 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:36:15.184977 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea202411-0a48-4098-b8f4-281e1ed8b0ca None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:15.286930 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea202411-0a48-4098-b8f4-281e1ed8b0ca None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:15.320304 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2bd145-6324-4858-90e5-955989596613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:15.321034 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2bd145-6324-4858-90e5-955989596613 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:15.321319 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2bd145-6324-4858-90e5-955989596613 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:15.321629 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2bd145-6324-4858-90e5-955989596613 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:15.322295 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2bd145-6324-4858-90e5-955989596613 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:15.323631 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2bd145-6324-4858-90e5-955989596613 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:15.336100 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-8c2bd145-6324-4858-90e5-955989596613 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 22:36:15.336100 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:15.336100 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:36:15.336100 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:36:15.336100 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:36:15.336100 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:36:15.336100 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:36:15.336100 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:36:15.336100 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 22:36:15.336100 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:36:15.336100 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:36:15.336100 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:36:15.336100 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:15.336100 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:15.336100 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:15.336100 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:15.336100 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:15.337048 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:15.337048 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:15.337048 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:15.337048 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:15.337048 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:15.337048 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:15.337048 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:36:15.337048 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:36:15.337048 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:36:15.337048 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:36:15.337048 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:15.337048 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:15.337048 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:36:15.337048 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:36:15.337048 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:36:15.337048 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:36:15.337915 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:36:15.337915 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:36:15.337915 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:36:15.337915 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:36:15.337915 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:36:15.337915 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:36:15.337915 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:36:15.337915 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:36:15.337915 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:36:15.337915 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:36:15.337915 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:36:15.337915 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:36:15.337915 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 22:36:15.337915 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:36:15.337915 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 505/1004] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Jul 21 22:36:15 2022] GET /identity/v3/services/object-store => generated 93 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:36:15.345013 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3689e5bb-d001-4b20-848c-180e0ac25554 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:15.449204 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3689e5bb-d001-4b20-848c-180e0ac25554 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:15.468518 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc930b-48bf-40d4-9d10-a76c2e836b79 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:15.468807 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc930b-48bf-40d4-9d10-a76c2e836b79 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:15.469082 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc930b-48bf-40d4-9d10-a76c2e836b79 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:15.470567 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 506/1005] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:36:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:36:15.477935 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908926cd-b169-4e84-b29b-ebf7326aca45 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:15.477935 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908926cd-b169-4e84-b29b-ebf7326aca45 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:15.477935 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908926cd-b169-4e84-b29b-ebf7326aca45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:15.490161 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe579cf3-9f40-4cc7-adc9-f2eacd21312b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:15.490869 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe579cf3-9f40-4cc7-adc9-f2eacd21312b None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:15.491159 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe579cf3-9f40-4cc7-adc9-f2eacd21312b None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:15.491438 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe579cf3-9f40-4cc7-adc9-f2eacd21312b None admin] PATH_INFO: `/v3/services` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:15.494677 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe579cf3-9f40-4cc7-adc9-f2eacd21312b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:15.494677 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe579cf3-9f40-4cc7-adc9-f2eacd21312b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:15.494677 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe579cf3-9f40-4cc7-adc9-f2eacd21312b None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:15.495613 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-908926cd-b169-4e84-b29b-ebf7326aca45 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:15.497227 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-908926cd-b169-4e84-b29b-ebf7326aca45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:15.502630 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 500/1006] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Jul 21 22:36:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:36:15.509267 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe579cf3-9f40-4cc7-adc9-f2eacd21312b None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:15.545355 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:15.555304 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:15.569715 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 507/1007] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:36:15 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:36:15.593532 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908926cd-b169-4e84-b29b-ebf7326aca45 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:15.623894 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe579cf3-9f40-4cc7-adc9-f2eacd21312b None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:15.662466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cda352-6c6f-4a81-8965-b23735b44498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:15.663293 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cda352-6c6f-4a81-8965-b23735b44498 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:15.663572 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cda352-6c6f-4a81-8965-b23735b44498 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:15.663888 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cda352-6c6f-4a81-8965-b23735b44498 None admin] PATH_INFO: `/v3/services` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:15.664588 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cda352-6c6f-4a81-8965-b23735b44498 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:15.664891 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cda352-6c6f-4a81-8965-b23735b44498 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:15.667128 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cda352-6c6f-4a81-8965-b23735b44498 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:15.676297 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 501/1008] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Jul 21 22:36:15 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:36:15.677253 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cda352-6c6f-4a81-8965-b23735b44498 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:15.699359 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908926cd-b169-4e84-b29b-ebf7326aca45 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:15.735163 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1770dd48-3407-4039-8ee5-ea1a779e8dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:15.735829 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1770dd48-3407-4039-8ee5-ea1a779e8dbb None admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:15.736113 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1770dd48-3407-4039-8ee5-ea1a779e8dbb None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:15.736393 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1770dd48-3407-4039-8ee5-ea1a779e8dbb None admin] PATH_INFO: `/v3/services` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:15.737120 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1770dd48-3407-4039-8ee5-ea1a779e8dbb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:15.738527 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1770dd48-3407-4039-8ee5-ea1a779e8dbb None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:15.774051 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:15.780854 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cda352-6c6f-4a81-8965-b23735b44498 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:15.814234 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc7f20-f49b-4f30-b889-c6a3aa5e3037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:15.814930 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc7f20-f49b-4f30-b889-c6a3aa5e3037 None admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:15.815220 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc7f20-f49b-4f30-b889-c6a3aa5e3037 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:15.815500 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc7f20-f49b-4f30-b889-c6a3aa5e3037 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:15.816281 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dc7f20-f49b-4f30-b889-c6a3aa5e3037 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:15.817640 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dc7f20-f49b-4f30-b889-c6a3aa5e3037 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:15.834766 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 508/1009] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jul 21 22:36:15 2022] POST /identity/v3/services => generated 242 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:36:15.912254 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 502/1010] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Jul 21 22:36:15 2022] POST /identity/v3/endpoints => generated 325 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:36:16.535253 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49651f0d-7347-4be0-8a8a-feb14f9c2f1a None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:16.535788 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49651f0d-7347-4be0-8a8a-feb14f9c2f1a None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:16.535788 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49651f0d-7347-4be0-8a8a-feb14f9c2f1a None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:16.537310 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 509/1011] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:36:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:36:16.542572 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6358bed-4bd1-448d-99b0-a2121ab673be None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:16.543026 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6358bed-4bd1-448d-99b0-a2121ab673be None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:16.543151 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6358bed-4bd1-448d-99b0-a2121ab673be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:16.560501 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-f6358bed-4bd1-448d-99b0-a2121ab673be None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:16.561684 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-f6358bed-4bd1-448d-99b0-a2121ab673be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:16.638401 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 503/1012] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:36:16 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:36:16.660774 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49651f0d-7347-4be0-8a8a-feb14f9c2f1a None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:16.712725 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:16.763538 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49651f0d-7347-4be0-8a8a-feb14f9c2f1a None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:16.797669 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55553ca-b8e6-4951-9903-6464156733f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:16.798450 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55553ca-b8e6-4951-9903-6464156733f3 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:16.798729 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55553ca-b8e6-4951-9903-6464156733f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:16.799027 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55553ca-b8e6-4951-9903-6464156733f3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:16.799665 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55553ca-b8e6-4951-9903-6464156733f3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:16.801566 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55553ca-b8e6-4951-9903-6464156733f3 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:16.812947 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-f55553ca-b8e6-4951-9903-6464156733f3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 22:36:16.812947 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:16.812947 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:36:16.812947 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:36:16.812947 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:36:16.812947 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:36:16.812947 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:36:16.812947 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:36:16.812947 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 22:36:16.812947 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:36:16.812947 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:36:16.812947 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:36:16.812947 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:16.812947 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:16.812947 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:16.812947 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:16.812947 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:16.813915 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:16.813915 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:16.813915 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:16.813915 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:16.813915 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:16.813915 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:16.813915 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:36:16.813915 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:36:16.813915 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:36:16.813915 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:36:16.813915 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:16.813915 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:16.813915 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:36:16.813915 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:36:16.813915 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:36:16.813915 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:36:16.820174 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:36:16.820174 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:36:16.820174 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:36:16.820174 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:36:16.820174 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:36:16.820174 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:36:16.820174 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:36:16.820174 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:36:16.820174 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:36:16.820174 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:36:16.820174 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:36:16.820174 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:36:16.820174 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 22:36:16.820174 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:36:16.820174 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 510/1013] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:36:16 2022] GET /identity/v3/services/compute => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:36:16.822520 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6358bed-4bd1-448d-99b0-a2121ab673be None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:16.902376 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:16.922809 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6358bed-4bd1-448d-99b0-a2121ab673be None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:16.958582 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77da5c5e-497b-473e-b9f0-c80646951039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:16.959404 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da5c5e-497b-473e-b9f0-c80646951039 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:16.959690 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da5c5e-497b-473e-b9f0-c80646951039 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:16.959969 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da5c5e-497b-473e-b9f0-c80646951039 None admin] PATH_INFO: `/v3/services` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:16.960702 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77da5c5e-497b-473e-b9f0-c80646951039 None admin] RBAC: Adding query filter params (name=compute) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:16.961010 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77da5c5e-497b-473e-b9f0-c80646951039 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:16.962413 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77da5c5e-497b-473e-b9f0-c80646951039 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:16.973848 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 504/1014] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Jul 21 22:36:16 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:36:16.975267 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55553ca-b8e6-4951-9903-6464156733f3 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:17.080761 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55553ca-b8e6-4951-9903-6464156733f3 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:17.093544 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:17.115522 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c8a8b0-4c50-4072-8ab2-0b8ea1b5f43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:17.116188 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c8a8b0-4c50-4072-8ab2-0b8ea1b5f43e None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:17.116471 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c8a8b0-4c50-4072-8ab2-0b8ea1b5f43e None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:17.116748 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c8a8b0-4c50-4072-8ab2-0b8ea1b5f43e None admin] PATH_INFO: `/v3/services` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:17.117453 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c8a8b0-4c50-4072-8ab2-0b8ea1b5f43e None admin] RBAC: Adding query filter params (type=compute) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:17.117759 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c8a8b0-4c50-4072-8ab2-0b8ea1b5f43e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:17.119209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c8a8b0-4c50-4072-8ab2-0b8ea1b5f43e None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:17.124140 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 511/1015] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Jul 21 22:36:16 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:17.129804 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77da5c5e-497b-473e-b9f0-c80646951039 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:17.236074 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77da5c5e-497b-473e-b9f0-c80646951039 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:17.269842 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ffb80-5a68-4fee-abb3-b18dedf2f589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:17.270487 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ffb80-5a68-4fee-abb3-b18dedf2f589 None admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:17.270781 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ffb80-5a68-4fee-abb3-b18dedf2f589 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:17.271066 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ffb80-5a68-4fee-abb3-b18dedf2f589 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:17.271810 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378ffb80-5a68-4fee-abb3-b18dedf2f589 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:17.273165 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378ffb80-5a68-4fee-abb3-b18dedf2f589 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:17.385920 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:17.398809 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 505/1016] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Jul 21 22:36:17 2022] POST /identity/v3/endpoints => generated 334 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:36:17.914235 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fba290-660b-4a26-9293-9da2e82865ef None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:17.914751 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fba290-660b-4a26-9293-9da2e82865ef None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:17.914857 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fba290-660b-4a26-9293-9da2e82865ef None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:17.916330 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 512/1017] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:36:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:36:17.921455 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154dd16-0e99-4fd9-adff-76f8dc1956a0 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:17.921832 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154dd16-0e99-4fd9-adff-76f8dc1956a0 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:17.922016 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154dd16-0e99-4fd9-adff-76f8dc1956a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:17.939184 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-7154dd16-0e99-4fd9-adff-76f8dc1956a0 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:17.940379 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-7154dd16-0e99-4fd9-adff-76f8dc1956a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:17.984361 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67f087e-cbe8-40b9-a6b7-b5edf36ab362 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:17.984678 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67f087e-cbe8-40b9-a6b7-b5edf36ab362 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:17.984924 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67f087e-cbe8-40b9-a6b7-b5edf36ab362 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:17.987035 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 513/1018] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:36:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:36:17.992522 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb26757c-d8b0-49d3-8bdf-730565ef4b0c None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:17.992810 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb26757c-d8b0-49d3-8bdf-730565ef4b0c None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:17.993096 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb26757c-d8b0-49d3-8bdf-730565ef4b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:18.009927 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-cb26757c-d8b0-49d3-8bdf-730565ef4b0c None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:18.011131 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-cb26757c-d8b0-49d3-8bdf-730565ef4b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:18.017716 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 506/1019] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:36:17 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:36:18.040009 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7154dd16-0e99-4fd9-adff-76f8dc1956a0 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:18.078210 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 514/1020] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:36:17 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:36:18.101396 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb26757c-d8b0-49d3-8bdf-730565ef4b0c None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:18.149530 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7154dd16-0e99-4fd9-adff-76f8dc1956a0 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:18.177042 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:18.185908 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fced03-c869-4a47-ab9d-29b738e5bf1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:18.186627 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fced03-c869-4a47-ab9d-29b738e5bf1a None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:18.187040 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fced03-c869-4a47-ab9d-29b738e5bf1a None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:18.187407 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fced03-c869-4a47-ab9d-29b738e5bf1a None admin] PATH_INFO: `/v3/services/image` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:18.188156 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fced03-c869-4a47-ab9d-29b738e5bf1a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:18.198982 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fced03-c869-4a47-ab9d-29b738e5bf1a None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:18.201650 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-12fced03-c869-4a47-ab9d-29b738e5bf1a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 22:36:18.201650 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:18.201650 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:36:18.201650 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:36:18.201650 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:36:18.201650 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:36:18.201650 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:36:18.201650 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:36:18.201650 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 22:36:18.201650 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:36:18.201650 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:36:18.201650 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:36:18.201650 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:18.201650 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:18.201650 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:18.201650 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:18.201650 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:18.202833 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:18.202833 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:18.202833 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:18.202833 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:18.202833 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:18.202833 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:18.202833 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:36:18.202833 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:36:18.202833 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:36:18.202833 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:36:18.202833 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:18.202833 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:18.202833 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:36:18.202833 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:36:18.202833 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:36:18.202833 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:36:18.203699 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:36:18.203699 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:36:18.203699 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:36:18.203699 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:36:18.203699 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:36:18.203699 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:36:18.203699 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:36:18.203699 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:36:18.203699 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:36:18.203699 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:36:18.203699 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:36:18.203699 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:36:18.203699 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 22:36:18.203699 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:36:18.203699 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 507/1021] 10.222.0.52 () {56 vars in 1105 bytes} [Thu Jul 21 22:36:18 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:36:18.208430 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fced03-c869-4a47-ab9d-29b738e5bf1a None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:18.210537 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb26757c-d8b0-49d3-8bdf-730565ef4b0c None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:18.219062 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:18.247411 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fd7582-af83-44cb-be29-e66fb91c1d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:18.248203 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd7582-af83-44cb-be29-e66fb91c1d6c None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:18.248484 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd7582-af83-44cb-be29-e66fb91c1d6c None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:18.248758 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd7582-af83-44cb-be29-e66fb91c1d6c None admin] PATH_INFO: `/v3/domains/default` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:18.249928 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fd7582-af83-44cb-be29-e66fb91c1d6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:18.251405 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fd7582-af83-44cb-be29-e66fb91c1d6c None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:18.258951 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 515/1022] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:36:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:36:18.259809 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fd7582-af83-44cb-be29-e66fb91c1d6c None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:18.324634 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fced03-c869-4a47-ab9d-29b738e5bf1a None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:18.361355 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecde47a-99c3-4c76-9dd2-d65b38658718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:18.362249 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecde47a-99c3-4c76-9dd2-d65b38658718 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:18.362684 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecde47a-99c3-4c76-9dd2-d65b38658718 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:18.363188 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecde47a-99c3-4c76-9dd2-d65b38658718 None admin] PATH_INFO: `/v3/services` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:18.364055 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecde47a-99c3-4c76-9dd2-d65b38658718 None admin] RBAC: Adding query filter params (name=image) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:18.364509 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecde47a-99c3-4c76-9dd2-d65b38658718 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:18.366032 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecde47a-99c3-4c76-9dd2-d65b38658718 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:18.366218 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fd7582-af83-44cb-be29-e66fb91c1d6c None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:18.371370 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 508/1023] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:36:18 2022] GET /identity/v3/services?name=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:36:18.376930 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecde47a-99c3-4c76-9dd2-d65b38658718 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:18.405138 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77a9b3b-e500-41c5-8cf7-8cb7fa3745b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:18.405792 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a9b3b-e500-41c5-8cf7-8cb7fa3745b3 None admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:18.406075 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a9b3b-e500-41c5-8cf7-8cb7fa3745b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:18.406345 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a9b3b-e500-41c5-8cf7-8cb7fa3745b3 None admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:18.407105 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77a9b3b-e500-41c5-8cf7-8cb7fa3745b3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:18.408510 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77a9b3b-e500-41c5-8cf7-8cb7fa3745b3 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:18.440107 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:18.482014 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecde47a-99c3-4c76-9dd2-d65b38658718 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:18.492741 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 516/1024] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jul 21 22:36:18 2022] POST /identity/v3/projects => generated 311 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:36:18.528329 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f3c92d-0547-4308-aa55-650f1e789ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:18.529026 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3c92d-0547-4308-aa55-650f1e789ce3 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:18.529309 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3c92d-0547-4308-aa55-650f1e789ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:18.529581 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3c92d-0547-4308-aa55-650f1e789ce3 None admin] PATH_INFO: `/v3/services` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:18.530510 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f3c92d-0547-4308-aa55-650f1e789ce3 None admin] RBAC: Adding query filter params (type=image) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:18.531052 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f3c92d-0547-4308-aa55-650f1e789ce3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:18.533632 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f3c92d-0547-4308-aa55-650f1e789ce3 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:18.545958 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 509/1025] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:36:18 2022] GET /identity/v3/services?type=image => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:36:18.547793 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77a9b3b-e500-41c5-8cf7-8cb7fa3745b3 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:18.579983 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:18.664382 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77a9b3b-e500-41c5-8cf7-8cb7fa3745b3 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:18.701108 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495cf719-04db-4ce8-bde5-284542353433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:18.702094 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495cf719-04db-4ce8-bde5-284542353433 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:18.702613 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495cf719-04db-4ce8-bde5-284542353433 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:18.702973 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495cf719-04db-4ce8-bde5-284542353433 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:18.703901 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495cf719-04db-4ce8-bde5-284542353433 None admin] RBAC: Adding query filter params (interface=public, service_id=547e994405764a60bff11cc24ca6c86e, region_id=RegionOne) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:18.704222 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495cf719-04db-4ce8-bde5-284542353433 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:18.705590 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495cf719-04db-4ce8-bde5-284542353433 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:18.712124 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 517/1026] 10.222.0.52 () {56 vars in 1251 bytes} [Thu Jul 21 22:36:18 2022] GET /identity/v3/endpoints?service_id=547e994405764a60bff11cc24ca6c86e&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:36:18.718013 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f3c92d-0547-4308-aa55-650f1e789ce3 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:18.820903 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f3c92d-0547-4308-aa55-650f1e789ce3 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:18.838084 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:18.855746 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38325e-61ff-4902-a7b1-5a684a8816dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:18.856410 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38325e-61ff-4902-a7b1-5a684a8816dc None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:18.856691 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38325e-61ff-4902-a7b1-5a684a8816dc None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:18.857033 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38325e-61ff-4902-a7b1-5a684a8816dc None admin] PATH_INFO: `/v3/services` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:18.857705 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be38325e-61ff-4902-a7b1-5a684a8816dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:18.859840 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be38325e-61ff-4902-a7b1-5a684a8816dc None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:18.870834 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 510/1027] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Jul 21 22:36:18 2022] GET /identity/v3/services => generated 1957 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:36:19.551882 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d823f-684f-47a5-942e-0c179650bb53 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:19.552416 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d823f-684f-47a5-942e-0c179650bb53 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:19.552416 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d823f-684f-47a5-942e-0c179650bb53 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:19.553998 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 518/1028] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:36:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:36:19.559193 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1372f2-5ed2-418f-981d-ee546c5c8d2d None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:19.559567 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1372f2-5ed2-418f-981d-ee546c5c8d2d None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:19.559741 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1372f2-5ed2-418f-981d-ee546c5c8d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:19.576785 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-af1372f2-5ed2-418f-981d-ee546c5c8d2d None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:19.577979 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-af1372f2-5ed2-418f-981d-ee546c5c8d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:19.650995 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 511/1029] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:36:19 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:36:19.675090 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d823f-684f-47a5-942e-0c179650bb53 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:19.768501 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:19.779847 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d823f-684f-47a5-942e-0c179650bb53 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:19.813930 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e0bb5-7cc7-4704-8d1e-60736010f852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:19.814738 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e0bb5-7cc7-4704-8d1e-60736010f852 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:19.815050 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e0bb5-7cc7-4704-8d1e-60736010f852 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:19.815326 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e0bb5-7cc7-4704-8d1e-60736010f852 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:19.825941 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22e0bb5-7cc7-4704-8d1e-60736010f852 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:36:19.834531 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-d22e0bb5-7cc7-4704-8d1e-60736010f852 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 22:36:19.834531 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:19.834531 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:19.834531 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:19.834531 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:19.834531 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:19.834531 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:19.834531 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:19.834531 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:19.834531 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:19.834531 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:19.834531 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:19.834531 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:36:19.834531 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:36:19.834531 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:36:19.834531 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:36:19.835543 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:36:19.835543 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:36:19.835543 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:36:19.835543 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:36:19.835543 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:19.835543 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:19.835543 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:36:19.835543 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:36:19.835543 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:36:19.835543 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:36:19.835543 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:36:19.835543 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:36:19.835543 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:36:19.835543 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:36:19.835543 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:36:19.835543 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:36:19.836394 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:36:19.836394 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:36:19.836394 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:36:19.836394 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:36:19.836394 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:36:19.836394 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:36:19.836394 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:36:19.836394 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:36:19.836394 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:36:19.836394 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:36:19.836394 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 22:36:19.836394 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:36:19.839096 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 519/1030] 10.222.0.52 () {56 vars in 1120 bytes} [Thu Jul 21 22:36:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:36:19.841770 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1372f2-5ed2-418f-981d-ee546c5c8d2d None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:19.942965 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1372f2-5ed2-418f-981d-ee546c5c8d2d None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:19.955965 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:19.979295 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594d5a2e-31c6-4dce-a08c-dd0c24403892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:19.979967 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d5a2e-31c6-4dce-a08c-dd0c24403892 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:19.980252 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d5a2e-31c6-4dce-a08c-dd0c24403892 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:19.980522 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d5a2e-31c6-4dce-a08c-dd0c24403892 None admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:19.981263 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594d5a2e-31c6-4dce-a08c-dd0c24403892 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:19.981565 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594d5a2e-31c6-4dce-a08c-dd0c24403892 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:19.983163 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594d5a2e-31c6-4dce-a08c-dd0c24403892 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:19.996691 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 512/1031] 10.222.0.52 () {56 vars in 1115 bytes} [Thu Jul 21 22:36:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:19.998566 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e0bb5-7cc7-4704-8d1e-60736010f852 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:20.099240 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e0bb5-7cc7-4704-8d1e-60736010f852 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:20.132803 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de76eb9a-4ab4-45f1-a14f-0c083289aeb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:20.133593 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76eb9a-4ab4-45f1-a14f-0c083289aeb9 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:20.133874 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76eb9a-4ab4-45f1-a14f-0c083289aeb9 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:20.134149 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76eb9a-4ab4-45f1-a14f-0c083289aeb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:20.145122 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de76eb9a-4ab4-45f1-a14f-0c083289aeb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:20.146547 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de76eb9a-4ab4-45f1-a14f-0c083289aeb9 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:20.163990 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-de76eb9a-4ab4-45f1-a14f-0c083289aeb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:36:20.163990 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:20.163990 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:20.163990 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:20.163990 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:20.163990 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:20.163990 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:20.163990 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:20.163990 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:20.163990 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:20.163990 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:20.163990 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:20.163990 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:36:20.163990 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:36:20.163990 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:36:20.164976 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:36:20.164976 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:20.164976 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:20.164976 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:36:20.164976 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:36:20.164976 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:36:20.164976 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:36:20.164976 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:36:20.164976 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:36:20.164976 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:36:20.164976 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:36:20.164976 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:36:20.164976 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:36:20.164976 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:36:20.164976 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:36:20.164976 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:36:20.164976 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:36:20.165867 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:36:20.165867 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:36:20.165867 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:36:20.165867 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:36:20.167097 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 520/1032] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:36:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:36:20.171009 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594d5a2e-31c6-4dce-a08c-dd0c24403892 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:20.271744 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594d5a2e-31c6-4dce-a08c-dd0c24403892 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:20.307495 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda7873a-fe51-4a8d-b1ed-f2ed4a53416b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:20.308160 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7873a-fe51-4a8d-b1ed-f2ed4a53416b None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:20.308437 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7873a-fe51-4a8d-b1ed-f2ed4a53416b None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:20.308823 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7873a-fe51-4a8d-b1ed-f2ed4a53416b None admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:20.309540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda7873a-fe51-4a8d-b1ed-f2ed4a53416b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:20.309842 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda7873a-fe51-4a8d-b1ed-f2ed4a53416b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:20.311290 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda7873a-fe51-4a8d-b1ed-f2ed4a53416b None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:20.327785 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 513/1033] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:36:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:20.329303 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de76eb9a-4ab4-45f1-a14f-0c083289aeb9 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:20.333028 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:20.439415 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de76eb9a-4ab4-45f1-a14f-0c083289aeb9 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:20.473390 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73537a07-8b28-40c7-ac5d-ede1c8a0582d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:20.474182 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73537a07-8b28-40c7-ac5d-ede1c8a0582d None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:20.474461 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73537a07-8b28-40c7-ac5d-ede1c8a0582d None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:20.474731 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73537a07-8b28-40c7-ac5d-ede1c8a0582d None admin] PATH_INFO: `/v3/users/nova` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:20.486888 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73537a07-8b28-40c7-ac5d-ede1c8a0582d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:20.488349 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73537a07-8b28-40c7-ac5d-ede1c8a0582d None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:20.504386 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-73537a07-8b28-40c7-ac5d-ede1c8a0582d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:36:20.504386 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:20.504386 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:20.504386 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:20.504386 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:20.504386 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:20.504386 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:20.504386 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:20.504386 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:20.504386 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:20.504386 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:20.504386 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:20.504386 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:36:20.504386 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:36:20.504386 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:36:20.504386 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:36:20.505335 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:20.505335 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:20.505335 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:36:20.505335 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:36:20.505335 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:36:20.505335 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:36:20.505335 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:36:20.505335 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:36:20.505335 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:36:20.505335 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:36:20.505335 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:36:20.505335 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:36:20.505335 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:36:20.505335 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:36:20.505335 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:36:20.505335 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:36:20.505335 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:36:20.505335 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:36:20.506264 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:36:20.506264 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:36:20.506264 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:36:20.506264 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:36:20.506264 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:36:20.506264 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:36:20.506264 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:36:20.506264 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:36:20.506264 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:36:20.506264 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:36:20.510252 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 521/1034] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Jul 21 22:36:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:36:20.511663 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda7873a-fe51-4a8d-b1ed-f2ed4a53416b None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:20.549354 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:20.613890 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd5a67-6e76-4128-a518-754e69682671 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:20.614429 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd5a67-6e76-4128-a518-754e69682671 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:20.614560 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd5a67-6e76-4128-a518-754e69682671 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:20.615937 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 522/1035] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:36:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:36:20.618749 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda7873a-fe51-4a8d-b1ed-f2ed4a53416b None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:20.620986 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83235861-499f-4d10-910d-3cc3f9aca294 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:20.621273 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83235861-499f-4d10-910d-3cc3f9aca294 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:20.621549 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83235861-499f-4d10-910d-3cc3f9aca294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:20.638955 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-83235861-499f-4d10-910d-3cc3f9aca294 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:20.640134 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-83235861-499f-4d10-910d-3cc3f9aca294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:20.661759 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9938cda-974f-455e-97f8-5aa40c4ec0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:20.662432 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9938cda-974f-455e-97f8-5aa40c4ec0bf None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:20.662713 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9938cda-974f-455e-97f8-5aa40c4ec0bf None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:20.663014 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9938cda-974f-455e-97f8-5aa40c4ec0bf None admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:20.663850 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9938cda-974f-455e-97f8-5aa40c4ec0bf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:20.664158 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9938cda-974f-455e-97f8-5aa40c4ec0bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:20.665573 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9938cda-974f-455e-97f8-5aa40c4ec0bf None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:20.683905 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 514/1036] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Jul 21 22:36:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:20.689244 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9938cda-974f-455e-97f8-5aa40c4ec0bf None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:20.702368 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:20.714939 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 523/1037] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:36:20 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:36:20.739207 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83235861-499f-4d10-910d-3cc3f9aca294 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:20.787284 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:20.796297 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9938cda-974f-455e-97f8-5aa40c4ec0bf None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:20.835626 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafd653d-869c-498d-a20d-960b85bdc38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:20.836278 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd653d-869c-498d-a20d-960b85bdc38d None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:20.836558 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd653d-869c-498d-a20d-960b85bdc38d None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:20.836833 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd653d-869c-498d-a20d-960b85bdc38d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:20.845196 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83235861-499f-4d10-910d-3cc3f9aca294 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:20.884693 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae00035-19d8-4991-913c-6753bf2b0985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:20.885853 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae00035-19d8-4991-913c-6753bf2b0985 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:20.886302 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae00035-19d8-4991-913c-6753bf2b0985 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:20.886751 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae00035-19d8-4991-913c-6753bf2b0985 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:20.888361 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae00035-19d8-4991-913c-6753bf2b0985 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:20.890002 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae00035-19d8-4991-913c-6753bf2b0985 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:20.897175 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 524/1038] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:36:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:36:20.898166 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae00035-19d8-4991-913c-6753bf2b0985 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:20.907342 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafd653d-869c-498d-a20d-960b85bdc38d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:20.908783 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafd653d-869c-498d-a20d-960b85bdc38d None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:20.963138 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:21.003565 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae00035-19d8-4991-913c-6753bf2b0985 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:21.004386 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:21.013623 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 515/1039] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:36:20 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 22:36:21.014785 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f04fd-dae8-4ec6-ae6f-b7967e3db382 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:21.015091 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f04fd-dae8-4ec6-ae6f-b7967e3db382 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:21.015429 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f04fd-dae8-4ec6-ae6f-b7967e3db382 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:21.016723 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 516/1040] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:36:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:36:21.021882 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13f04fd-dae8-4ec6-ae6f-b7967e3db382 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:21.040509 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0273f046-f458-4e15-ba94-a4da54726cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:21.041180 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0273f046-f458-4e15-ba94-a4da54726cc0 None admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:21.041467 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0273f046-f458-4e15-ba94-a4da54726cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:21.041742 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0273f046-f458-4e15-ba94-a4da54726cc0 None admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:21.042448 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0273f046-f458-4e15-ba94-a4da54726cc0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:21.043909 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0273f046-f458-4e15-ba94-a4da54726cc0 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:21.120859 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13f04fd-dae8-4ec6-ae6f-b7967e3db382 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:21.153149 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ae87f-6079-4ca4-adf9-f71f135b8a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:21.153942 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ae87f-6079-4ca4-adf9-f71f135b8a94 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:21.154222 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ae87f-6079-4ca4-adf9-f71f135b8a94 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:21.154491 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ae87f-6079-4ca4-adf9-f71f135b8a94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:21.161499 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 525/1041] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 21 22:36:20 2022] POST /identity/v3/users => generated 285 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:36:21.163120 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55927c80-6c70-4d76-b46f-372f9e29afb6 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:21.163407 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55927c80-6c70-4d76-b46f-372f9e29afb6 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:21.163990 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55927c80-6c70-4d76-b46f-372f9e29afb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:21.166206 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24ae87f-6079-4ca4-adf9-f71f135b8a94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:21.167719 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24ae87f-6079-4ca4-adf9-f71f135b8a94 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:21.182240 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-55927c80-6c70-4d76-b46f-372f9e29afb6 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:21.183500 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-55927c80-6c70-4d76-b46f-372f9e29afb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:21.188624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-e24ae87f-6079-4ca4-adf9-f71f135b8a94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:36:21.188624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:21.188624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:21.188624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:21.188624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:21.188624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:21.188624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:21.188624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:21.188624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:21.188624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:21.188624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:21.188624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:21.188624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:36:21.188624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:36:21.188624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:36:21.189625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:36:21.189625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:21.189625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:21.189625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:36:21.189625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:36:21.189625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:36:21.189625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:36:21.189625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:36:21.189625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:36:21.189625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:36:21.189625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:36:21.189625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:36:21.189625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:36:21.189625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:36:21.189625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:36:21.189625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:36:21.189625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:36:21.190613 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:36:21.190613 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:36:21.190613 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:36:21.190613 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:36:21.190613 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 517/1042] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:36:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:36:21.196672 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ae87f-6079-4ca4-adf9-f71f135b8a94 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:21.248342 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:21.268686 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 526/1043] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:36:21 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:36:21.294273 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55927c80-6c70-4d76-b46f-372f9e29afb6 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:21.315154 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ae87f-6079-4ca4-adf9-f71f135b8a94 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:21.346818 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:21.356160 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cab65e-782f-413c-aff9-ccf663b541b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:21.356823 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cab65e-782f-413c-aff9-ccf663b541b9 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:21.357104 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cab65e-782f-413c-aff9-ccf663b541b9 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:21.359274 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cab65e-782f-413c-aff9-ccf663b541b9 None admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:21.359274 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cab65e-782f-413c-aff9-ccf663b541b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:21.359274 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cab65e-782f-413c-aff9-ccf663b541b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:21.359808 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cab65e-782f-413c-aff9-ccf663b541b9 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:21.374289 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 518/1044] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:36:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:36:21.380522 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cab65e-782f-413c-aff9-ccf663b541b9 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:21.414893 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55927c80-6c70-4d76-b46f-372f9e29afb6 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:21.450522 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924416a6-3b1d-47ac-9d34-9acf7fe78586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:21.451349 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924416a6-3b1d-47ac-9d34-9acf7fe78586 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:21.451636 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924416a6-3b1d-47ac-9d34-9acf7fe78586 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:21.451911 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924416a6-3b1d-47ac-9d34-9acf7fe78586 None admin] PATH_INFO: `/v3/services/image` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:21.452556 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924416a6-3b1d-47ac-9d34-9acf7fe78586 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:21.453917 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924416a6-3b1d-47ac-9d34-9acf7fe78586 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:21.466323 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-924416a6-3b1d-47ac-9d34-9acf7fe78586 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 22:36:21.466323 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:21.466323 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:36:21.466323 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:36:21.466323 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:36:21.466323 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:36:21.466323 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:36:21.466323 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:36:21.466323 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 22:36:21.466323 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:36:21.466323 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:36:21.466323 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:36:21.466323 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:21.466323 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:21.466323 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:21.466323 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:21.466323 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:21.467465 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:21.467465 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:21.467465 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:21.467465 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:21.467465 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:21.467465 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:21.467465 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:36:21.467465 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:36:21.467465 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:36:21.467465 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:36:21.467465 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:21.467465 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:21.467465 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:36:21.467465 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:36:21.467465 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:36:21.467465 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:36:21.468405 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:36:21.468405 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:36:21.468405 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:36:21.468405 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:36:21.468405 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:36:21.468405 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:36:21.468405 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:36:21.468405 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:36:21.468405 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:36:21.468405 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:36:21.468405 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:36:21.468405 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:36:21.468405 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 22:36:21.468405 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:36:21.468405 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 527/1045] 10.222.0.52 () {56 vars in 1105 bytes} [Thu Jul 21 22:36:21 2022] GET /identity/v3/services/image => generated 86 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:36:21.473269 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924416a6-3b1d-47ac-9d34-9acf7fe78586 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:21.500280 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cab65e-782f-413c-aff9-ccf663b541b9 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:21.502636 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:21.537251 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f12b047-f68e-44e3-a53b-10e07860a065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:21.538050 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f12b047-f68e-44e3-a53b-10e07860a065 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:21.538328 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f12b047-f68e-44e3-a53b-10e07860a065 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:21.538604 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f12b047-f68e-44e3-a53b-10e07860a065 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:21.545464 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f12b047-f68e-44e3-a53b-10e07860a065 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:21.546918 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f12b047-f68e-44e3-a53b-10e07860a065 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:21.561074 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-6f12b047-f68e-44e3-a53b-10e07860a065 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:36:21.561074 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:21.561074 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:21.561074 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:21.561074 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:21.561074 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:21.561074 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:21.561074 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:21.561074 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:21.561074 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:21.561074 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:21.561074 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:21.561074 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:36:21.561074 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:36:21.561074 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:36:21.562014 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:36:21.562014 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:21.562014 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:21.562014 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:36:21.562014 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:36:21.562014 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:36:21.562014 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:36:21.562014 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:36:21.562014 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:36:21.562014 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:36:21.562014 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:36:21.562014 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:36:21.562014 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:36:21.562014 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:36:21.562014 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:36:21.562014 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:36:21.562014 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:36:21.562990 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:36:21.562990 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:36:21.562990 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:36:21.562990 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:36:21.562990 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:36:21.562990 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:36:21.562990 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:36:21.562990 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:36:21.562990 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 519/1046] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:36:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:36:21.568987 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f12b047-f68e-44e3-a53b-10e07860a065 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:21.588858 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924416a6-3b1d-47ac-9d34-9acf7fe78586 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:21.599811 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:21.627790 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6e99d1-f7f8-48ed-bd69-2d119a3c3df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:21.628787 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e99d1-f7f8-48ed-bd69-2d119a3c3df7 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:21.629441 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e99d1-f7f8-48ed-bd69-2d119a3c3df7 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:21.630093 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e99d1-f7f8-48ed-bd69-2d119a3c3df7 None admin] PATH_INFO: `/v3/services` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:21.631713 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6e99d1-f7f8-48ed-bd69-2d119a3c3df7 None admin] RBAC: Adding query filter params (name=image) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:21.632448 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6e99d1-f7f8-48ed-bd69-2d119a3c3df7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:21.634724 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6e99d1-f7f8-48ed-bd69-2d119a3c3df7 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:21.645957 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 528/1047] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:36:21 2022] GET /identity/v3/services?name=image => generated 122 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:21.647929 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6e99d1-f7f8-48ed-bd69-2d119a3c3df7 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:21.679852 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f12b047-f68e-44e3-a53b-10e07860a065 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:21.714968 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a358f92-5275-4a66-ae56-8e3ef657de05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:21.715620 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a358f92-5275-4a66-ae56-8e3ef657de05 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:21.715906 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a358f92-5275-4a66-ae56-8e3ef657de05 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:21.716178 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a358f92-5275-4a66-ae56-8e3ef657de05 None admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:21.716956 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a358f92-5275-4a66-ae56-8e3ef657de05 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:21.717259 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a358f92-5275-4a66-ae56-8e3ef657de05 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:21.718670 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a358f92-5275-4a66-ae56-8e3ef657de05 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:21.732036 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 520/1048] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 22:36:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:21.738098 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a358f92-5275-4a66-ae56-8e3ef657de05 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:21.759633 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6e99d1-f7f8-48ed-bd69-2d119a3c3df7 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:21.765132 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:21.795436 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930e7869-7975-48cb-9dc9-6505850df47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:21.796093 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e7869-7975-48cb-9dc9-6505850df47f None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:21.796377 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e7869-7975-48cb-9dc9-6505850df47f None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:21.796651 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e7869-7975-48cb-9dc9-6505850df47f None admin] PATH_INFO: `/v3/services` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:21.797350 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930e7869-7975-48cb-9dc9-6505850df47f None admin] RBAC: Adding query filter params (type=image) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:21.797652 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930e7869-7975-48cb-9dc9-6505850df47f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:21.799033 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930e7869-7975-48cb-9dc9-6505850df47f None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:21.805085 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 529/1049] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:36:21 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:21.810381 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930e7869-7975-48cb-9dc9-6505850df47f None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:21.848696 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a358f92-5275-4a66-ae56-8e3ef657de05 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:21.852221 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:21.886232 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fd8e12-7ac5-4ebc-8a82-c103872e5edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:21.886924 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd8e12-7ac5-4ebc-8a82-c103872e5edd None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:21.887207 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd8e12-7ac5-4ebc-8a82-c103872e5edd None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:21.887478 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd8e12-7ac5-4ebc-8a82-c103872e5edd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:21.888304 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fd8e12-7ac5-4ebc-8a82-c103872e5edd None admin] RBAC: Adding query filter params (role.id=6a667c5557a5498f943c65b692078b57, scope.project.id=aa11ab8229b642ce962193b74001d3bc, user.id=6bfdb655a2a44b33ab724daaf73a294b) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:21.888611 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fd8e12-7ac5-4ebc-8a82-c103872e5edd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:21.890435 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fd8e12-7ac5-4ebc-8a82-c103872e5edd None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:21.906801 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 521/1050] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 22:36:21 2022] GET /identity/v3/role_assignments?user.id=6bfdb655a2a44b33ab724daaf73a294b&scope.project.id=aa11ab8229b642ce962193b74001d3bc&role.id=6a667c5557a5498f943c65b692078b57 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:36:21.928344 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930e7869-7975-48cb-9dc9-6505850df47f None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:21.966122 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100d015e-790d-43f8-b35c-49a70642b384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:21.966825 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100d015e-790d-43f8-b35c-49a70642b384 None admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:21.967114 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100d015e-790d-43f8-b35c-49a70642b384 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:21.967392 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100d015e-790d-43f8-b35c-49a70642b384 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:21.968154 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100d015e-790d-43f8-b35c-49a70642b384 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:21.969549 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100d015e-790d-43f8-b35c-49a70642b384 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:22.048993 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 530/1051] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Jul 21 22:36:21 2022] POST /identity/v3/endpoints => generated 327 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:36:23.352538 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d1c5fd-4a6b-493c-8117-aae8eca8a753 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:23.353095 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d1c5fd-4a6b-493c-8117-aae8eca8a753 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:23.353175 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d1c5fd-4a6b-493c-8117-aae8eca8a753 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:23.354663 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 522/1052] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:36:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:36:23.359963 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756932d-7b94-44f6-8449-918865367baa None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:23.360356 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756932d-7b94-44f6-8449-918865367baa None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:23.360518 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756932d-7b94-44f6-8449-918865367baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:23.380444 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-4756932d-7b94-44f6-8449-918865367baa None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:23.381717 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-4756932d-7b94-44f6-8449-918865367baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:23.467823 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 531/1053] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:36:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:36:23.493939 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d1c5fd-4a6b-493c-8117-aae8eca8a753 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:23.581179 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:23.594508 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d1c5fd-4a6b-493c-8117-aae8eca8a753 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:23.634694 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0039d5-2d65-48e5-b25d-927b5f64b66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:23.635908 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0039d5-2d65-48e5-b25d-927b5f64b66f None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:23.636433 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0039d5-2d65-48e5-b25d-927b5f64b66f None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:23.637354 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0039d5-2d65-48e5-b25d-927b5f64b66f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:23.650742 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0039d5-2d65-48e5-b25d-927b5f64b66f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:36:23.658641 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-ab0039d5-2d65-48e5-b25d-927b5f64b66f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:36:23.658641 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:23.658641 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:23.658641 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:23.658641 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:23.658641 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:23.658641 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:23.658641 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:23.658641 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:23.658641 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:23.658641 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:23.658641 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:23.658641 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:36:23.658641 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:36:23.658641 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:36:23.658641 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:36:23.661258 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:36:23.661258 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:36:23.661258 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:36:23.661258 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:36:23.661258 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:23.661258 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:23.661258 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:36:23.661258 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:36:23.661258 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:36:23.661258 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:36:23.661258 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:36:23.661258 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:36:23.661258 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:36:23.661258 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:36:23.661258 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:36:23.661258 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:36:23.662135 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:36:23.662135 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:36:23.662135 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:36:23.662135 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:36:23.662135 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:36:23.662135 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:36:23.662135 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:36:23.662135 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:36:23.662135 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:36:23.662135 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:36:23.662135 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:36:23.662135 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:36:23.666011 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 523/1054] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 22:36:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:36:23.667175 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4756932d-7b94-44f6-8449-918865367baa None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:23.690209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:23.771041 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4756932d-7b94-44f6-8449-918865367baa None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:23.806430 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c5240-b54d-4f81-adec-4bd6fdb39c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:23.807142 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c5240-b54d-4f81-adec-4bd6fdb39c7b None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:23.807429 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c5240-b54d-4f81-adec-4bd6fdb39c7b None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:23.807704 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c5240-b54d-4f81-adec-4bd6fdb39c7b None admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:23.808451 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42c5240-b54d-4f81-adec-4bd6fdb39c7b None admin] RBAC: Adding query filter params (name=admin) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:23.808758 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42c5240-b54d-4f81-adec-4bd6fdb39c7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:23.810119 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42c5240-b54d-4f81-adec-4bd6fdb39c7b None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:23.823373 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 532/1055] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:36:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:23.824574 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0039d5-2d65-48e5-b25d-927b5f64b66f None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:23.924289 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0c61a-f46e-4fdf-812f-49176faa1c2e None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:23.924559 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0c61a-f46e-4fdf-812f-49176faa1c2e None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:23.925396 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0c61a-f46e-4fdf-812f-49176faa1c2e None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:23.926679 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 533/1056] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:36:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:36:23.929910 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0039d5-2d65-48e5-b25d-927b5f64b66f None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:23.931664 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da839772-9a6f-40de-9bae-99ee419eaabf None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:23.931957 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da839772-9a6f-40de-9bae-99ee419eaabf None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:23.932245 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da839772-9a6f-40de-9bae-99ee419eaabf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:23.949585 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-da839772-9a6f-40de-9bae-99ee419eaabf None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:23.950820 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-da839772-9a6f-40de-9bae-99ee419eaabf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:23.971552 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0bc4e2-169f-41ca-ad9f-5bc2ee9705ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:23.972501 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0bc4e2-169f-41ca-ad9f-5bc2ee9705ea None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:23.972816 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0bc4e2-169f-41ca-ad9f-5bc2ee9705ea None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:23.973863 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0bc4e2-169f-41ca-ad9f-5bc2ee9705ea None admin] PATH_INFO: `/v3/users/ba2177d2a0914eba9e11b306859f6f5f` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:23.990004 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0bc4e2-169f-41ca-ad9f-5bc2ee9705ea None admin] RBAC: Authorizing `identity:get_user(user_id=ba2177d2a0914eba9e11b306859f6f5f)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:23.991477 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0bc4e2-169f-41ca-ad9f-5bc2ee9705ea None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:23.997769 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 524/1057] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 22:36:23 2022] GET /identity/v3/users/ba2177d2a0914eba9e11b306859f6f5f => generated 285 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:36:23.998703 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0bc4e2-169f-41ca-ad9f-5bc2ee9705ea None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:24.008053 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:24.029006 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 534/1058] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:36:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:36:24.052141 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da839772-9a6f-40de-9bae-99ee419eaabf None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:24.067336 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:24.105443 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0bc4e2-169f-41ca-ad9f-5bc2ee9705ea None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:24.146835 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221706ff-58f8-4600-89a1-c663f97005da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:24.148335 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221706ff-58f8-4600-89a1-c663f97005da None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:24.148777 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221706ff-58f8-4600-89a1-c663f97005da None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:24.149476 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221706ff-58f8-4600-89a1-c663f97005da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:24.156060 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da839772-9a6f-40de-9bae-99ee419eaabf None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:24.191438 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598a1e70-5d76-4b88-9233-3cfbfe34895e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:24.192171 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a1e70-5d76-4b88-9233-3cfbfe34895e None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:24.192450 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a1e70-5d76-4b88-9233-3cfbfe34895e None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:24.192724 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a1e70-5d76-4b88-9233-3cfbfe34895e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:24.202622 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598a1e70-5d76-4b88-9233-3cfbfe34895e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:36:24.210796 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-598a1e70-5d76-4b88-9233-3cfbfe34895e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 22:36:24.210796 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:24.210796 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:24.210796 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:24.210796 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:24.210796 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:24.210796 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:24.210796 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:24.210796 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:24.210796 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:24.210796 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:24.210796 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:24.210796 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:36:24.210796 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:36:24.210796 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:36:24.210796 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:36:24.211740 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:36:24.211740 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:36:24.211740 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:36:24.211740 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:36:24.211740 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:24.211740 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:24.211740 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:36:24.211740 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:36:24.211740 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:36:24.211740 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:36:24.211740 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:36:24.211740 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:36:24.211740 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:36:24.211740 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:36:24.211740 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:36:24.211740 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:36:24.212581 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:36:24.212581 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:36:24.212581 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:36:24.212581 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:36:24.212581 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:36:24.212581 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:36:24.212581 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:36:24.212581 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:36:24.212581 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:36:24.212581 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:36:24.212581 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 22:36:24.212581 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:36:24.212581 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 535/1059] 10.222.0.52 () {56 vars in 1120 bytes} [Thu Jul 21 22:36:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:36:24.213286 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8667ec-1edf-4a0b-9dc3-9b716034b98a None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:24.213733 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8667ec-1edf-4a0b-9dc3-9b716034b98a None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:24.213733 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8667ec-1edf-4a0b-9dc3-9b716034b98a None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:24.215181 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 536/1060] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:36:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:36:24.217724 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8667ec-1edf-4a0b-9dc3-9b716034b98a None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:24.222305 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221706ff-58f8-4600-89a1-c663f97005da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:24.223781 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221706ff-58f8-4600-89a1-c663f97005da None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:24.231976 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:24.291365 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:24.322189 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8667ec-1edf-4a0b-9dc3-9b716034b98a None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:24.327967 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 525/1061] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:36:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 22:36:24.329265 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed599cf-c177-4122-af13-74811f07a7da None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:24.329555 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed599cf-c177-4122-af13-74811f07a7da None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:24.329830 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed599cf-c177-4122-af13-74811f07a7da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:24.345491 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-3ed599cf-c177-4122-af13-74811f07a7da None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:24.346670 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-3ed599cf-c177-4122-af13-74811f07a7da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:24.356197 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47268e7-7790-4f93-8efc-0df1c61bd494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:24.356859 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47268e7-7790-4f93-8efc-0df1c61bd494 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:24.357137 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47268e7-7790-4f93-8efc-0df1c61bd494 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:24.357410 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47268e7-7790-4f93-8efc-0df1c61bd494 None admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:24.358192 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47268e7-7790-4f93-8efc-0df1c61bd494 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:24.358496 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47268e7-7790-4f93-8efc-0df1c61bd494 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:24.359885 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47268e7-7790-4f93-8efc-0df1c61bd494 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:24.368245 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 537/1062] 10.222.0.52 () {56 vars in 1115 bytes} [Thu Jul 21 22:36:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:36:24.369018 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47268e7-7790-4f93-8efc-0df1c61bd494 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:24.416613 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 526/1063] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:36:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:36:24.417454 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed599cf-c177-4122-af13-74811f07a7da None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:24.452593 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:24.475082 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47268e7-7790-4f93-8efc-0df1c61bd494 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:24.511282 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4207a837-8a86-45ab-b2a4-25ba6c490894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:24.512134 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4207a837-8a86-45ab-b2a4-25ba6c490894 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:24.512415 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4207a837-8a86-45ab-b2a4-25ba6c490894 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:24.512688 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4207a837-8a86-45ab-b2a4-25ba6c490894 None admin] PATH_INFO: `/v3/projects/4e8aa015f53442c1add8861248076f46` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:24.513850 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4207a837-8a86-45ab-b2a4-25ba6c490894 None admin] RBAC: Authorizing `identity:get_project(project_id=4e8aa015f53442c1add8861248076f46)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:24.515297 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4207a837-8a86-45ab-b2a4-25ba6c490894 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:24.516983 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 538/1064] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:36:24 2022] GET /identity/v3/projects/4e8aa015f53442c1add8861248076f46 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:24.517636 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed599cf-c177-4122-af13-74811f07a7da None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:24.517747 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4207a837-8a86-45ab-b2a4-25ba6c490894 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:24.521000 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:24.554454 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e8c8e3-541e-460c-a476-4f865be947b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:24.555292 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e8c8e3-541e-460c-a476-4f865be947b0 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:24.555579 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e8c8e3-541e-460c-a476-4f865be947b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:24.555856 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e8c8e3-541e-460c-a476-4f865be947b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:24.566364 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e8c8e3-541e-460c-a476-4f865be947b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:24.567852 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e8c8e3-541e-460c-a476-4f865be947b0 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:24.586297 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-f2e8c8e3-541e-460c-a476-4f865be947b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:36:24.586297 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:24.586297 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:24.586297 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:24.586297 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:24.586297 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:24.586297 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:24.586297 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:24.586297 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:24.586297 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:24.586297 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:24.586297 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:24.586297 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:36:24.586297 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:36:24.586297 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:36:24.587300 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:36:24.587300 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:24.587300 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:24.587300 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:36:24.587300 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:36:24.587300 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:36:24.587300 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:36:24.587300 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:36:24.587300 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:36:24.587300 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:36:24.587300 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:36:24.587300 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:36:24.587300 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:36:24.587300 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:36:24.587300 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:36:24.587300 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:36:24.587300 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:36:24.588185 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:36:24.588185 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:36:24.588185 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:36:24.588185 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:36:24.588185 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 527/1065] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:36:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:36:24.588480 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e8c8e3-541e-460c-a476-4f865be947b0 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:24.630827 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4207a837-8a86-45ab-b2a4-25ba6c490894 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:24.664990 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a9843-dc0d-4fce-b1d9-07741736b093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:24.665736 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a9843-dc0d-4fce-b1d9-07741736b093 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:24.666021 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a9843-dc0d-4fce-b1d9-07741736b093 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:24.666297 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a9843-dc0d-4fce-b1d9-07741736b093 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:24.676164 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:24.687811 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e8c8e3-541e-460c-a476-4f865be947b0 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:24.721757 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f069dab-b631-4722-8bd1-97bdbb6bc4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:24.722860 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f069dab-b631-4722-8bd1-97bdbb6bc4ee None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:24.723141 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f069dab-b631-4722-8bd1-97bdbb6bc4ee None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:24.723414 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f069dab-b631-4722-8bd1-97bdbb6bc4ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:24.724240 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f069dab-b631-4722-8bd1-97bdbb6bc4ee None admin] RBAC: Adding query filter params (role.id=d1ddee590a964ff1bbb4ee8716173021, scope.project.id=4e8aa015f53442c1add8861248076f46, user.id=ba2177d2a0914eba9e11b306859f6f5f) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:24.724550 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f069dab-b631-4722-8bd1-97bdbb6bc4ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:24.726238 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f069dab-b631-4722-8bd1-97bdbb6bc4ee None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:24.731696 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 528/1066] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 22:36:24 2022] GET /identity/v3/role_assignments?user.id=ba2177d2a0914eba9e11b306859f6f5f&scope.project.id=4e8aa015f53442c1add8861248076f46&role.id=d1ddee590a964ff1bbb4ee8716173021 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:24.732456 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f069dab-b631-4722-8bd1-97bdbb6bc4ee None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:24.733428 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494a9843-dc0d-4fce-b1d9-07741736b093 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:24.734917 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494a9843-dc0d-4fce-b1d9-07741736b093 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:24.737011 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:24.851373 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f069dab-b631-4722-8bd1-97bdbb6bc4ee None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:24.853230 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 539/1067] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:36:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 22:36:24.859425 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a9843-dc0d-4fce-b1d9-07741736b093 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:24.897633 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bcb4c3-a9e4-4111-a8cc-8979194d9083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:24.898312 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bcb4c3-a9e4-4111-a8cc-8979194d9083 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:24.898598 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bcb4c3-a9e4-4111-a8cc-8979194d9083 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:24.898921 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bcb4c3-a9e4-4111-a8cc-8979194d9083 None admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:24.899650 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bcb4c3-a9e4-4111-a8cc-8979194d9083 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:24.900056 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bcb4c3-a9e4-4111-a8cc-8979194d9083 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:24.901542 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bcb4c3-a9e4-4111-a8cc-8979194d9083 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:24.918935 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 529/1068] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:36:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:24.924659 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bcb4c3-a9e4-4111-a8cc-8979194d9083 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:24.941874 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:24.973725 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a9843-dc0d-4fce-b1d9-07741736b093 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:24.976314 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:25.011458 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b852d70-c774-4cfd-8d23-57027daeb6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:25.012783 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b852d70-c774-4cfd-8d23-57027daeb6a1 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:25.013073 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b852d70-c774-4cfd-8d23-57027daeb6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:25.013377 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b852d70-c774-4cfd-8d23-57027daeb6a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:25.024531 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b852d70-c774-4cfd-8d23-57027daeb6a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:25.026077 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b852d70-c774-4cfd-8d23-57027daeb6a1 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:25.033452 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bcb4c3-a9e4-4111-a8cc-8979194d9083 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:25.045559 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-7b852d70-c774-4cfd-8d23-57027daeb6a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:36:25.045559 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:25.045559 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:25.045559 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:25.045559 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:25.045559 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:25.045559 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:25.045559 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:25.045559 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:25.045559 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:25.045559 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:25.045559 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:25.045559 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:36:25.045559 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:36:25.045559 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:36:25.046566 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:36:25.046566 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:25.046566 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:25.046566 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:36:25.046566 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:36:25.046566 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:36:25.046566 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:36:25.046566 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:36:25.046566 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:36:25.046566 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:36:25.046566 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:36:25.046566 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:36:25.046566 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:36:25.046566 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:36:25.046566 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:36:25.046566 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:36:25.046566 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:36:25.047544 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:36:25.047544 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:36:25.047544 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:36:25.047544 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:36:25.047544 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 540/1069] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:36:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:36:25.053570 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b852d70-c774-4cfd-8d23-57027daeb6a1 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:25.076258 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42761bee-83f2-4d57-9ac5-9217cd224856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:25.077080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42761bee-83f2-4d57-9ac5-9217cd224856 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:25.077346 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42761bee-83f2-4d57-9ac5-9217cd224856 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:25.077650 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42761bee-83f2-4d57-9ac5-9217cd224856 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:25.085137 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42761bee-83f2-4d57-9ac5-9217cd224856 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:25.086566 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42761bee-83f2-4d57-9ac5-9217cd224856 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:25.102382 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-42761bee-83f2-4d57-9ac5-9217cd224856 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:36:25.102382 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:25.102382 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:25.102382 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:25.102382 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:25.102382 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:25.102382 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:25.102382 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:25.102382 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:25.102382 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:25.102382 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:25.102382 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:25.102382 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:36:25.102382 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:36:25.102382 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:36:25.102382 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:36:25.103508 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:25.103508 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:25.103508 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:36:25.103508 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:36:25.103508 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:36:25.103508 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:36:25.103508 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:36:25.103508 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:36:25.103508 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:36:25.103508 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:36:25.103508 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:36:25.103508 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:36:25.103508 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:36:25.103508 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:36:25.103508 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:36:25.103508 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:36:25.103508 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:36:25.103508 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:36:25.104797 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:36:25.104797 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:36:25.104797 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:36:25.104797 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:36:25.104797 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:36:25.104797 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:36:25.104797 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:36:25.104797 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:36:25.104797 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:36:25.104797 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:36:25.104797 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 530/1070] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Jul 21 22:36:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:36:25.109329 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42761bee-83f2-4d57-9ac5-9217cd224856 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:25.165955 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b852d70-c774-4cfd-8d23-57027daeb6a1 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:25.189273 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:25.203356 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd28b686-7994-4eb8-b6b5-d90003b1b168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:25.204029 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd28b686-7994-4eb8-b6b5-d90003b1b168 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:25.204414 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd28b686-7994-4eb8-b6b5-d90003b1b168 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:25.204584 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd28b686-7994-4eb8-b6b5-d90003b1b168 None admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:25.205343 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd28b686-7994-4eb8-b6b5-d90003b1b168 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:25.205606 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd28b686-7994-4eb8-b6b5-d90003b1b168 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:25.207575 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd28b686-7994-4eb8-b6b5-d90003b1b168 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:25.220112 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 541/1071] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:36:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:36:25.222384 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42761bee-83f2-4d57-9ac5-9217cd224856 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:25.264876 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baabe7e4-1418-45df-bd1f-c2eae0ac107e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:25.265944 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baabe7e4-1418-45df-bd1f-c2eae0ac107e None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:25.266431 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baabe7e4-1418-45df-bd1f-c2eae0ac107e None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:25.266945 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baabe7e4-1418-45df-bd1f-c2eae0ac107e None admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:25.268029 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baabe7e4-1418-45df-bd1f-c2eae0ac107e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:25.268644 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baabe7e4-1418-45df-bd1f-c2eae0ac107e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:25.270956 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baabe7e4-1418-45df-bd1f-c2eae0ac107e None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:25.291682 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 531/1072] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Jul 21 22:36:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:36:25.297236 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd28b686-7994-4eb8-b6b5-d90003b1b168 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:25.315320 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:25.408513 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd28b686-7994-4eb8-b6b5-d90003b1b168 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:25.448202 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc51de8c-2dee-4a15-bf9c-c4a3a2a611a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:25.449002 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51de8c-2dee-4a15-bf9c-c4a3a2a611a6 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:25.449309 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51de8c-2dee-4a15-bf9c-c4a3a2a611a6 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:25.449582 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51de8c-2dee-4a15-bf9c-c4a3a2a611a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:25.461806 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc51de8c-2dee-4a15-bf9c-c4a3a2a611a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:25.463308 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc51de8c-2dee-4a15-bf9c-c4a3a2a611a6 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:25.482048 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-dc51de8c-2dee-4a15-bf9c-c4a3a2a611a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:36:25.482048 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:25.482048 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:25.482048 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:25.482048 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:25.482048 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:25.482048 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:25.482048 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:25.482048 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:25.482048 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:25.482048 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:25.482048 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:25.482048 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:36:25.482048 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:36:25.482048 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:36:25.483064 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:36:25.483064 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:25.483064 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:25.483064 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:36:25.483064 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:36:25.483064 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:36:25.483064 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:36:25.483064 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:36:25.483064 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:36:25.483064 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:36:25.483064 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:36:25.483064 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:36:25.483064 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:36:25.483064 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:36:25.483064 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:36:25.483064 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:36:25.483064 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:36:25.484203 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:36:25.484203 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:36:25.484203 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:36:25.484203 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:36:25.484203 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 542/1073] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:36:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:36:25.489660 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baabe7e4-1418-45df-bd1f-c2eae0ac107e None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:25.592200 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baabe7e4-1418-45df-bd1f-c2eae0ac107e None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:25.620402 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:25.626016 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3ed85-76c9-48fb-b1a5-ca7c5290e554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:25.626675 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3ed85-76c9-48fb-b1a5-ca7c5290e554 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:25.627030 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3ed85-76c9-48fb-b1a5-ca7c5290e554 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:25.627313 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3ed85-76c9-48fb-b1a5-ca7c5290e554 None admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:25.628684 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3ed85-76c9-48fb-b1a5-ca7c5290e554 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:25.628974 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3ed85-76c9-48fb-b1a5-ca7c5290e554 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:25.630339 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3ed85-76c9-48fb-b1a5-ca7c5290e554 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:25.641415 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 532/1074] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:36:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:36:25.647129 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc51de8c-2dee-4a15-bf9c-c4a3a2a611a6 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:25.712500 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:25.747799 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc51de8c-2dee-4a15-bf9c-c4a3a2a611a6 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:25.780806 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd3b4f-2869-4989-a3b2-fb1f6471b5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:25.781601 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd3b4f-2869-4989-a3b2-fb1f6471b5e0 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:25.781888 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd3b4f-2869-4989-a3b2-fb1f6471b5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:25.782167 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd3b4f-2869-4989-a3b2-fb1f6471b5e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:25.789380 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dd3b4f-2869-4989-a3b2-fb1f6471b5e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:25.790844 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dd3b4f-2869-4989-a3b2-fb1f6471b5e0 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:25.807637 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-a8dd3b4f-2869-4989-a3b2-fb1f6471b5e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:36:25.807637 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:25.807637 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:25.807637 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:25.807637 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:25.807637 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:25.807637 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:25.807637 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:25.807637 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:25.807637 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:25.807637 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:25.807637 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:25.807637 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:36:25.807637 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:36:25.807637 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:36:25.808712 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:36:25.808712 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:25.808712 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:25.808712 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:36:25.808712 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:36:25.808712 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:36:25.808712 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:36:25.808712 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:36:25.808712 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:36:25.808712 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:36:25.808712 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:36:25.808712 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:36:25.808712 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:36:25.808712 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:36:25.808712 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:36:25.808712 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:36:25.808712 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:36:25.814314 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:36:25.814314 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:36:25.814314 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:36:25.814314 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:36:25.814314 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:36:25.814314 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:36:25.814314 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:36:25.814314 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:36:25.814314 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 543/1075] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:36:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:36:25.814828 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3ed85-76c9-48fb-b1a5-ca7c5290e554 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:25.914315 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3ed85-76c9-48fb-b1a5-ca7c5290e554 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:25.946578 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb41e5f-d06d-4dd1-9ff8-f9b8a535f894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:25.947259 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb41e5f-d06d-4dd1-9ff8-f9b8a535f894 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:25.947538 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb41e5f-d06d-4dd1-9ff8-f9b8a535f894 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:25.947811 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb41e5f-d06d-4dd1-9ff8-f9b8a535f894 None admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:25.948586 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb41e5f-d06d-4dd1-9ff8-f9b8a535f894 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:25.948894 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb41e5f-d06d-4dd1-9ff8-f9b8a535f894 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:25.950304 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb41e5f-d06d-4dd1-9ff8-f9b8a535f894 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:25.961920 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 533/1076] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 22:36:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:25.967913 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd3b4f-2869-4989-a3b2-fb1f6471b5e0 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:26.064565 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd3b4f-2869-4989-a3b2-fb1f6471b5e0 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:26.088701 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:26.097278 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a795e-e285-4362-83a7-0db9a9685e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:26.098169 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a795e-e285-4362-83a7-0db9a9685e60 None admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:26.098447 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a795e-e285-4362-83a7-0db9a9685e60 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:26.098715 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a795e-e285-4362-83a7-0db9a9685e60 None admin] PATH_INFO: `/v3/projects/aa11ab8229b642ce962193b74001d3bc/users/6bfdb655a2a44b33ab724daaf73a294b/roles/6a667c5557a5498f943c65b692078b57` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:26.101145 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512a795e-e285-4362-83a7-0db9a9685e60 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa11ab8229b642ce962193b74001d3bc, user_id=6bfdb655a2a44b33ab724daaf73a294b, role_id=6a667c5557a5498f943c65b692078b57)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:26.102580 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512a795e-e285-4362-83a7-0db9a9685e60 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:26.154732 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 544/1077] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 21 22:36:25 2022] PUT /identity/v3/projects/aa11ab8229b642ce962193b74001d3bc/users/6bfdb655a2a44b33ab724daaf73a294b/roles/6a667c5557a5498f943c65b692078b57 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:36:26.764153 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4a485-19dc-4841-b10b-a14943949ee6 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:26.764645 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4a485-19dc-4841-b10b-a14943949ee6 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:26.764645 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4a485-19dc-4841-b10b-a14943949ee6 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:26.766135 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 534/1078] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:36:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:36:26.771153 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee144c76-7bd5-4160-926c-9896274e2fc5 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:26.771530 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee144c76-7bd5-4160-926c-9896274e2fc5 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:26.771813 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee144c76-7bd5-4160-926c-9896274e2fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:26.788853 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-ee144c76-7bd5-4160-926c-9896274e2fc5 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:26.790007 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-ee144c76-7bd5-4160-926c-9896274e2fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:26.856428 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 545/1079] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:36:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:36:26.879038 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c4a485-19dc-4841-b10b-a14943949ee6 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:26.930027 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:26.978754 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c4a485-19dc-4841-b10b-a14943949ee6 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:27.011871 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e4d342-34c5-4e73-a6d3-550bcc2a42a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:27.012647 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e4d342-34c5-4e73-a6d3-550bcc2a42a2 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:27.012929 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e4d342-34c5-4e73-a6d3-550bcc2a42a2 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:27.013202 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e4d342-34c5-4e73-a6d3-550bcc2a42a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:27.023129 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e4d342-34c5-4e73-a6d3-550bcc2a42a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:36:27.030514 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-53e4d342-34c5-4e73-a6d3-550bcc2a42a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:36:27.030514 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:27.030514 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:27.030514 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:27.030514 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:27.030514 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:27.030514 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:27.030514 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:27.030514 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:27.030514 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:27.030514 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:27.030514 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:27.030514 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:36:27.030514 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:36:27.030514 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:36:27.030514 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:36:27.031643 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:36:27.031643 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:36:27.031643 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:36:27.031643 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:36:27.031643 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:27.031643 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:27.031643 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:36:27.031643 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:36:27.031643 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:36:27.031643 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:36:27.031643 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:36:27.031643 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:36:27.031643 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:36:27.031643 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:36:27.031643 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:36:27.031643 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:36:27.037389 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:36:27.037389 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:36:27.037389 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:36:27.037389 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:36:27.037389 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:36:27.037389 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:36:27.037389 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:36:27.037389 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:36:27.037389 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:36:27.037389 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:36:27.037389 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:36:27.037389 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:36:27.037389 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 535/1080] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 22:36:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:36:27.038103 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee144c76-7bd5-4160-926c-9896274e2fc5 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:27.133781 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee144c76-7bd5-4160-926c-9896274e2fc5 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:27.149391 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:27.166378 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e399c14d-ad96-450d-b0ba-d3542ad6256d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:27.167078 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399c14d-ad96-450d-b0ba-d3542ad6256d None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:27.167358 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399c14d-ad96-450d-b0ba-d3542ad6256d None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:27.167631 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399c14d-ad96-450d-b0ba-d3542ad6256d None admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:27.168363 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e399c14d-ad96-450d-b0ba-d3542ad6256d None admin] RBAC: Adding query filter params (name=admin) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:27.168666 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e399c14d-ad96-450d-b0ba-d3542ad6256d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:27.170011 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e399c14d-ad96-450d-b0ba-d3542ad6256d None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:27.178678 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 546/1081] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:36:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:36:27.184572 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e4d342-34c5-4e73-a6d3-550bcc2a42a2 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:27.286896 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e4d342-34c5-4e73-a6d3-550bcc2a42a2 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:27.296969 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:27.319948 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cb15db-5103-4e1b-ab61-259d42384275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:27.320816 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb15db-5103-4e1b-ab61-259d42384275 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:27.321098 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb15db-5103-4e1b-ab61-259d42384275 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:27.321370 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb15db-5103-4e1b-ab61-259d42384275 None admin] PATH_INFO: `/v3/users/ba2177d2a0914eba9e11b306859f6f5f` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:27.322549 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cb15db-5103-4e1b-ab61-259d42384275 None admin] RBAC: Authorizing `identity:get_user(user_id=ba2177d2a0914eba9e11b306859f6f5f)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:27.324003 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cb15db-5103-4e1b-ab61-259d42384275 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:27.325848 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 536/1082] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 22:36:27 2022] GET /identity/v3/users/ba2177d2a0914eba9e11b306859f6f5f => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:27.331999 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e399c14d-ad96-450d-b0ba-d3542ad6256d None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:27.431367 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e399c14d-ad96-450d-b0ba-d3542ad6256d None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:27.464215 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c65683-d4cf-4bc7-a3a6-a22549c1d9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:27.465057 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c65683-d4cf-4bc7-a3a6-a22549c1d9fb None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:27.465335 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c65683-d4cf-4bc7-a3a6-a22549c1d9fb None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:27.465609 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c65683-d4cf-4bc7-a3a6-a22549c1d9fb None admin] PATH_INFO: `/v3/projects/4e8aa015f53442c1add8861248076f46` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:27.466791 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c65683-d4cf-4bc7-a3a6-a22549c1d9fb None admin] RBAC: Authorizing `identity:get_project(project_id=4e8aa015f53442c1add8861248076f46)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:27.468192 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c65683-d4cf-4bc7-a3a6-a22549c1d9fb None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:27.469844 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 547/1083] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:36:27 2022] GET /identity/v3/projects/4e8aa015f53442c1add8861248076f46 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:27.476190 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cb15db-5103-4e1b-ab61-259d42384275 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:27.577411 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cb15db-5103-4e1b-ab61-259d42384275 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:27.610655 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa6a1bd-d72d-40af-a081-c0fb58ff7c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:27.611579 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa6a1bd-d72d-40af-a081-c0fb58ff7c70 None admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:27.611859 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa6a1bd-d72d-40af-a081-c0fb58ff7c70 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:27.612131 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa6a1bd-d72d-40af-a081-c0fb58ff7c70 None admin] PATH_INFO: `/v3/projects/4e8aa015f53442c1add8861248076f46/users/ba2177d2a0914eba9e11b306859f6f5f/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:27.614102 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa6a1bd-d72d-40af-a081-c0fb58ff7c70 None admin] RBAC: Authorizing `identity:create_grant(project_id=4e8aa015f53442c1add8861248076f46, user_id=ba2177d2a0914eba9e11b306859f6f5f, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:27.615540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa6a1bd-d72d-40af-a081-c0fb58ff7c70 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:27.698534 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 537/1084] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 21 22:36:27 2022] PUT /identity/v3/projects/4e8aa015f53442c1add8861248076f46/users/ba2177d2a0914eba9e11b306859f6f5f/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:36:28.195214 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d0962-757b-4507-8182-9d48f03c3f3f None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:28.195736 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d0962-757b-4507-8182-9d48f03c3f3f None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:28.195826 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d0962-757b-4507-8182-9d48f03c3f3f None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:28.197120 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 548/1085] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:36:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:36:28.202410 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8447a38-f8c2-4969-9e26-cbecef2f01be None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:28.202833 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8447a38-f8c2-4969-9e26-cbecef2f01be None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:28.203140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8447a38-f8c2-4969-9e26-cbecef2f01be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:28.219429 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-e8447a38-f8c2-4969-9e26-cbecef2f01be None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:28.220587 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-e8447a38-f8c2-4969-9e26-cbecef2f01be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:28.250006 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:28.285643 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 538/1086] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:36:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:36:28.309154 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266d0962-757b-4507-8182-9d48f03c3f3f None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:28.356735 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:28.411784 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266d0962-757b-4507-8182-9d48f03c3f3f None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:28.444666 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf1a2c3-1748-4a2c-8db0-cf97a97b22c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:28.445524 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf1a2c3-1748-4a2c-8db0-cf97a97b22c6 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:28.445754 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf1a2c3-1748-4a2c-8db0-cf97a97b22c6 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:28.446026 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf1a2c3-1748-4a2c-8db0-cf97a97b22c6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:28.457052 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf1a2c3-1748-4a2c-8db0-cf97a97b22c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:36:28.465044 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-2bf1a2c3-1748-4a2c-8db0-cf97a97b22c6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 22:36:28.465044 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:28.465044 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:28.465044 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:28.465044 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:28.465044 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:28.465044 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:28.465044 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:28.465044 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:28.465044 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:28.465044 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:28.465044 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:28.465044 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:36:28.465044 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:36:28.465044 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:36:28.465044 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:36:28.466135 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:36:28.466135 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:36:28.466135 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:36:28.466135 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:36:28.466135 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:28.466135 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:28.466135 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:36:28.466135 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:36:28.466135 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:36:28.466135 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:36:28.466135 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:36:28.466135 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:36:28.466135 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:36:28.466135 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:36:28.466135 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:36:28.466135 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:36:28.467055 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:36:28.467055 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:36:28.467055 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:36:28.467055 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:36:28.467055 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:36:28.467055 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:36:28.467055 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:36:28.467055 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:36:28.467055 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:36:28.467055 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:36:28.467055 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 22:36:28.467055 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:36:28.467055 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 549/1087] 10.222.0.52 () {56 vars in 1120 bytes} [Thu Jul 21 22:36:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:36:28.472792 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8447a38-f8c2-4969-9e26-cbecef2f01be None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:28.572788 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8447a38-f8c2-4969-9e26-cbecef2f01be None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:28.606689 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff89bd2-f174-4a03-a54f-83dfa9587fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:28.607375 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff89bd2-f174-4a03-a54f-83dfa9587fcb None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:28.607655 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff89bd2-f174-4a03-a54f-83dfa9587fcb None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:28.607931 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff89bd2-f174-4a03-a54f-83dfa9587fcb None admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:28.608710 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff89bd2-f174-4a03-a54f-83dfa9587fcb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:28.609016 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff89bd2-f174-4a03-a54f-83dfa9587fcb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:28.610394 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff89bd2-f174-4a03-a54f-83dfa9587fcb None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:28.618210 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 539/1088] 10.222.0.52 () {56 vars in 1115 bytes} [Thu Jul 21 22:36:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:28.623877 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf1a2c3-1748-4a2c-8db0-cf97a97b22c6 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:28.721511 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf1a2c3-1748-4a2c-8db0-cf97a97b22c6 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:28.729405 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:28.755223 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f0c5bf-d72e-466c-b211-b613de211b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:28.756016 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f0c5bf-d72e-466c-b211-b613de211b1d None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:28.756302 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f0c5bf-d72e-466c-b211-b613de211b1d None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:28.756577 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f0c5bf-d72e-466c-b211-b613de211b1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:28.767435 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f0c5bf-d72e-466c-b211-b613de211b1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:28.769422 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f0c5bf-d72e-466c-b211-b613de211b1d None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:28.787497 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-22f0c5bf-d72e-466c-b211-b613de211b1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:36:28.787497 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:28.787497 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:28.787497 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:28.787497 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:28.787497 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:28.787497 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:28.787497 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:28.787497 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:28.787497 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:28.787497 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:28.787497 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:28.787497 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:36:28.787497 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:36:28.787497 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:36:28.788627 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:36:28.788627 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:28.788627 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:28.788627 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:36:28.788627 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:36:28.788627 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:36:28.788627 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:36:28.788627 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:36:28.788627 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:36:28.788627 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:36:28.788627 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:36:28.788627 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:36:28.788627 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:36:28.788627 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:36:28.788627 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:36:28.788627 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:36:28.788627 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:36:28.789650 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:36:28.789650 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:36:28.789650 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:36:28.789650 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:36:28.789650 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 550/1089] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:36:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:36:28.795019 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff89bd2-f174-4a03-a54f-83dfa9587fcb None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:28.825816 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:28.894432 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff89bd2-f174-4a03-a54f-83dfa9587fcb None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:28.933843 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ad3274-51d5-4b72-aa9b-1e1256bc77af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:28.934505 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad3274-51d5-4b72-aa9b-1e1256bc77af None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:28.934813 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad3274-51d5-4b72-aa9b-1e1256bc77af None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:28.935096 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad3274-51d5-4b72-aa9b-1e1256bc77af None admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:28.935822 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ad3274-51d5-4b72-aa9b-1e1256bc77af None admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:28.936131 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ad3274-51d5-4b72-aa9b-1e1256bc77af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:28.937495 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ad3274-51d5-4b72-aa9b-1e1256bc77af None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:28.950695 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 540/1090] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:36:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:28.956938 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f0c5bf-d72e-466c-b211-b613de211b1d None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:29.053420 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f0c5bf-d72e-466c-b211-b613de211b1d None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:29.086125 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e796516-c5e6-4e31-90a8-a0f94d071983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:29.086919 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e796516-c5e6-4e31-90a8-a0f94d071983 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:29.087199 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e796516-c5e6-4e31-90a8-a0f94d071983 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:29.087470 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e796516-c5e6-4e31-90a8-a0f94d071983 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:29.094646 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e796516-c5e6-4e31-90a8-a0f94d071983 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:29.096083 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e796516-c5e6-4e31-90a8-a0f94d071983 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:29.109761 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-1e796516-c5e6-4e31-90a8-a0f94d071983 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:36:29.109761 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:29.109761 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:29.109761 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:29.109761 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:29.109761 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:29.109761 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:29.109761 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:29.109761 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:29.109761 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:29.109761 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:29.109761 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:29.109761 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:36:29.109761 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:36:29.109761 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:36:29.109761 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:36:29.110904 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:29.110904 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:29.110904 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:36:29.110904 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:36:29.110904 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:36:29.110904 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:36:29.110904 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:36:29.110904 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:36:29.110904 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:36:29.110904 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:36:29.110904 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:36:29.110904 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:36:29.110904 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:36:29.110904 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:36:29.110904 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:36:29.110904 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:36:29.110904 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:36:29.110904 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:36:29.111879 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:36:29.111879 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:36:29.111879 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:36:29.111879 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:36:29.111879 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:36:29.111879 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:36:29.111879 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:36:29.111879 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:36:29.111879 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 22:36:29.111879 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:36:29.111879 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 551/1091] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Jul 21 22:36:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:36:29.116792 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ad3274-51d5-4b72-aa9b-1e1256bc77af None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:29.205786 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:29.215157 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ad3274-51d5-4b72-aa9b-1e1256bc77af None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:29.249185 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aabc32-9395-45ca-8cb7-26d1dfe501bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:29.249854 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aabc32-9395-45ca-8cb7-26d1dfe501bb None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:29.250136 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aabc32-9395-45ca-8cb7-26d1dfe501bb None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:29.250408 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aabc32-9395-45ca-8cb7-26d1dfe501bb None admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:29.251266 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aabc32-9395-45ca-8cb7-26d1dfe501bb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:29.251576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aabc32-9395-45ca-8cb7-26d1dfe501bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:29.253605 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aabc32-9395-45ca-8cb7-26d1dfe501bb None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:29.270989 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 541/1092] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Jul 21 22:36:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:29.276676 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e796516-c5e6-4e31-90a8-a0f94d071983 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:29.296755 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:29.376980 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e796516-c5e6-4e31-90a8-a0f94d071983 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:29.410716 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f77be2-b1af-416c-be33-e4ec8158f601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:29.411388 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f77be2-b1af-416c-be33-e4ec8158f601 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:29.411676 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f77be2-b1af-416c-be33-e4ec8158f601 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:29.411954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f77be2-b1af-416c-be33-e4ec8158f601 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:29.480123 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f77be2-b1af-416c-be33-e4ec8158f601 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:29.481585 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f77be2-b1af-416c-be33-e4ec8158f601 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:29.503744 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:29.584641 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 552/1093] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:36:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 22:36:29.590479 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aabc32-9395-45ca-8cb7-26d1dfe501bb None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:29.688793 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aabc32-9395-45ca-8cb7-26d1dfe501bb None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:29.722538 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd062723-a4b9-4bc0-b4b0-b2abc3759815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:29.723361 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd062723-a4b9-4bc0-b4b0-b2abc3759815 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:29.723643 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd062723-a4b9-4bc0-b4b0-b2abc3759815 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:29.723917 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd062723-a4b9-4bc0-b4b0-b2abc3759815 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:29.733408 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707fcc2f-5558-4d90-9647-1343ec1d782c None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:29.733701 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707fcc2f-5558-4d90-9647-1343ec1d782c None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:29.733985 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707fcc2f-5558-4d90-9647-1343ec1d782c None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:29.734214 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd062723-a4b9-4bc0-b4b0-b2abc3759815 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:29.735377 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 553/1094] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:36:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:36:29.735745 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd062723-a4b9-4bc0-b4b0-b2abc3759815 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:29.740205 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c03e1e-f60e-4184-829c-520313cfae0f None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:29.740493 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c03e1e-f60e-4184-829c-520313cfae0f None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:29.740769 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c03e1e-f60e-4184-829c-520313cfae0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:29.744204 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:29.754229 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-fd062723-a4b9-4bc0-b4b0-b2abc3759815 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:36:29.754229 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:29.754229 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:29.754229 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:29.754229 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:29.754229 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:29.754229 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:29.754229 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:29.754229 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:29.754229 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:29.754229 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:29.754229 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:29.754229 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:36:29.754229 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:36:29.754229 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:36:29.760286 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:36:29.760286 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:29.760286 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:29.760286 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:36:29.760286 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:36:29.760286 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:36:29.760286 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:36:29.760286 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:36:29.760286 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:36:29.760286 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:36:29.760286 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:36:29.760286 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:36:29.760286 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:36:29.760286 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:36:29.760286 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:36:29.760286 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:36:29.760286 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:36:29.761223 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:36:29.761223 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:36:29.761223 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:36:29.761223 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:36:29.761223 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 542/1095] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:36:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:36:29.761512 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-c2c03e1e-f60e-4184-829c-520313cfae0f None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:29.761512 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-c2c03e1e-f60e-4184-829c-520313cfae0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:29.761642 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd062723-a4b9-4bc0-b4b0-b2abc3759815 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:29.824458 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 554/1096] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:36:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:36:29.847768 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c03e1e-f60e-4184-829c-520313cfae0f None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:29.860664 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd062723-a4b9-4bc0-b4b0-b2abc3759815 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:29.894947 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e378fb-30bf-4555-b0a5-c8d3675418cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:29.895252 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:29.895773 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e378fb-30bf-4555-b0a5-c8d3675418cb None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:29.896062 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e378fb-30bf-4555-b0a5-c8d3675418cb None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:29.896339 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e378fb-30bf-4555-b0a5-c8d3675418cb None admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:29.897061 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e378fb-30bf-4555-b0a5-c8d3675418cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:29.897368 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e378fb-30bf-4555-b0a5-c8d3675418cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:29.898734 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e378fb-30bf-4555-b0a5-c8d3675418cb None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:29.910101 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 543/1097] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:36:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:29.915644 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e378fb-30bf-4555-b0a5-c8d3675418cb None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:29.946605 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c03e1e-f60e-4184-829c-520313cfae0f None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:29.973172 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:29.979357 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33430161-6e41-4c60-829a-3b2bcde48908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:29.980148 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33430161-6e41-4c60-829a-3b2bcde48908 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:29.980430 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33430161-6e41-4c60-829a-3b2bcde48908 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:29.980703 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33430161-6e41-4c60-829a-3b2bcde48908 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:29.991458 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33430161-6e41-4c60-829a-3b2bcde48908 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:36:29.999459 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-33430161-6e41-4c60-829a-3b2bcde48908 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:36:29.999459 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:29.999459 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:29.999459 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:29.999459 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:29.999459 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:29.999459 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:29.999459 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:29.999459 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:29.999459 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:29.999459 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:29.999459 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:29.999459 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:36:29.999459 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:36:29.999459 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:36:29.999459 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:36:30.000457 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:36:30.000457 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:36:30.000457 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:36:30.000457 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:36:30.000457 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:30.000457 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:30.000457 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:36:30.000457 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:36:30.000457 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:36:30.000457 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:36:30.000457 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:36:30.000457 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:36:30.000457 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:36:30.000457 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:36:30.000457 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:36:30.000457 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:36:30.001315 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:36:30.001315 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:36:30.001315 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:36:30.001315 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:36:30.001315 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:36:30.001315 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:36:30.001315 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:36:30.001315 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:36:30.001315 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:36:30.001315 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:36:30.001315 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:36:30.001315 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:36:30.003352 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 555/1098] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 22:36:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:36:30.007133 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33430161-6e41-4c60-829a-3b2bcde48908 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:30.015852 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e378fb-30bf-4555-b0a5-c8d3675418cb None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:30.050628 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52248c83-7793-46a7-82ba-2ca93341d2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:30.051439 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52248c83-7793-46a7-82ba-2ca93341d2d4 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:30.051716 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52248c83-7793-46a7-82ba-2ca93341d2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:30.051983 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52248c83-7793-46a7-82ba-2ca93341d2d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:30.058712 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52248c83-7793-46a7-82ba-2ca93341d2d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:30.060155 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52248c83-7793-46a7-82ba-2ca93341d2d4 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:30.075386 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-52248c83-7793-46a7-82ba-2ca93341d2d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:36:30.075386 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:30.075386 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:30.075386 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:30.075386 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:30.075386 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:30.075386 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:30.075386 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:30.075386 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:30.075386 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:30.075386 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:30.075386 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:30.075386 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:36:30.075386 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:36:30.075386 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:36:30.076356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:36:30.076356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:30.076356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:30.076356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:36:30.076356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:36:30.076356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:36:30.076356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:36:30.076356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:36:30.076356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:36:30.076356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:36:30.076356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:36:30.076356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:36:30.076356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:36:30.076356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:36:30.076356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:36:30.076356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:36:30.076356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:36:30.077372 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:36:30.077372 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:36:30.077372 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:36:30.077372 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:36:30.077372 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:36:30.077372 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:36:30.077372 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:36:30.077372 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:36:30.077372 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 544/1099] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:36:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:36:30.082295 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52248c83-7793-46a7-82ba-2ca93341d2d4 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:30.108299 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33430161-6e41-4c60-829a-3b2bcde48908 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:30.117618 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:30.142791 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61bad87-7c00-4370-b3da-f0186b898eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:30.143592 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61bad87-7c00-4370-b3da-f0186b898eb8 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:30.143895 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61bad87-7c00-4370-b3da-f0186b898eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:30.144448 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61bad87-7c00-4370-b3da-f0186b898eb8 None admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:30.145193 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61bad87-7c00-4370-b3da-f0186b898eb8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:30.145502 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61bad87-7c00-4370-b3da-f0186b898eb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:30.146925 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61bad87-7c00-4370-b3da-f0186b898eb8 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:30.155458 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 556/1100] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:36:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:36:30.161232 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61bad87-7c00-4370-b3da-f0186b898eb8 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:30.179938 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52248c83-7793-46a7-82ba-2ca93341d2d4 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:30.200120 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:30.214231 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda7e12d-6143-4d90-a3a5-4e1ee957771c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:30.214999 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda7e12d-6143-4d90-a3a5-4e1ee957771c None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:30.215280 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda7e12d-6143-4d90-a3a5-4e1ee957771c None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:30.215553 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda7e12d-6143-4d90-a3a5-4e1ee957771c None admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:30.216336 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda7e12d-6143-4d90-a3a5-4e1ee957771c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:30.216646 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda7e12d-6143-4d90-a3a5-4e1ee957771c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:30.218058 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda7e12d-6143-4d90-a3a5-4e1ee957771c None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:30.230139 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 545/1101] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 22:36:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:36:30.236068 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda7e12d-6143-4d90-a3a5-4e1ee957771c None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:30.260283 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61bad87-7c00-4370-b3da-f0186b898eb8 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:30.293731 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac1c720-49dc-4a33-ae5f-a37aa090a897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:30.294597 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1c720-49dc-4a33-ae5f-a37aa090a897 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:30.294909 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1c720-49dc-4a33-ae5f-a37aa090a897 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:30.295186 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1c720-49dc-4a33-ae5f-a37aa090a897 None admin] PATH_INFO: `/v3/users/ba2177d2a0914eba9e11b306859f6f5f` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:30.296370 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac1c720-49dc-4a33-ae5f-a37aa090a897 None admin] RBAC: Authorizing `identity:get_user(user_id=ba2177d2a0914eba9e11b306859f6f5f)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:30.297786 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac1c720-49dc-4a33-ae5f-a37aa090a897 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:30.299427 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 557/1102] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 22:36:30 2022] GET /identity/v3/users/ba2177d2a0914eba9e11b306859f6f5f => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:36:30.305510 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac1c720-49dc-4a33-ae5f-a37aa090a897 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:30.334684 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda7e12d-6143-4d90-a3a5-4e1ee957771c None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:30.344242 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:30.369154 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c642ed47-f16a-40d2-9fda-5a0d0c8d9648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:30.369804 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c642ed47-f16a-40d2-9fda-5a0d0c8d9648 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:30.370097 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c642ed47-f16a-40d2-9fda-5a0d0c8d9648 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:30.370365 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c642ed47-f16a-40d2-9fda-5a0d0c8d9648 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:30.371230 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c642ed47-f16a-40d2-9fda-5a0d0c8d9648 None admin] RBAC: Adding query filter params (role.id=6a667c5557a5498f943c65b692078b57, scope.project.id=aa11ab8229b642ce962193b74001d3bc, user.id=6bfdb655a2a44b33ab724daaf73a294b) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:30.371544 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c642ed47-f16a-40d2-9fda-5a0d0c8d9648 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:30.372963 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c642ed47-f16a-40d2-9fda-5a0d0c8d9648 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:30.389809 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 546/1103] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 22:36:30 2022] GET /identity/v3/role_assignments?user.id=6bfdb655a2a44b33ab724daaf73a294b&scope.project.id=aa11ab8229b642ce962193b74001d3bc&role.id=6a667c5557a5498f943c65b692078b57 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:36:30.408414 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac1c720-49dc-4a33-ae5f-a37aa090a897 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:30.446135 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58638864-879a-4dfa-afe6-80b75a9e241b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:30.446837 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58638864-879a-4dfa-afe6-80b75a9e241b None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:30.447149 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58638864-879a-4dfa-afe6-80b75a9e241b None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:30.447408 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58638864-879a-4dfa-afe6-80b75a9e241b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:30.521738 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58638864-879a-4dfa-afe6-80b75a9e241b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:30.523238 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58638864-879a-4dfa-afe6-80b75a9e241b None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:30.619767 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:30.681207 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 558/1104] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:36:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 22:36:30.687341 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c642ed47-f16a-40d2-9fda-5a0d0c8d9648 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:30.737445 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:30.792620 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c642ed47-f16a-40d2-9fda-5a0d0c8d9648 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:30.828189 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fd3664-1e1b-4415-a60a-268d2f3dee3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:30.829046 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd3664-1e1b-4415-a60a-268d2f3dee3c None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:30.829332 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd3664-1e1b-4415-a60a-268d2f3dee3c None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:30.829612 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd3664-1e1b-4415-a60a-268d2f3dee3c None admin] PATH_INFO: `/v3/projects/4e8aa015f53442c1add8861248076f46` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:30.830823 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fd3664-1e1b-4415-a60a-268d2f3dee3c None admin] RBAC: Authorizing `identity:get_project(project_id=4e8aa015f53442c1add8861248076f46)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:30.832259 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fd3664-1e1b-4415-a60a-268d2f3dee3c None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:30.834271 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 547/1105] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:36:30 2022] GET /identity/v3/projects/4e8aa015f53442c1add8861248076f46 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:30.840228 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58638864-879a-4dfa-afe6-80b75a9e241b None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:30.941851 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58638864-879a-4dfa-afe6-80b75a9e241b None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:30.983369 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e54dcb-55fc-4a5e-8ad2-92ffc0bfd3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:30.984037 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e54dcb-55fc-4a5e-8ad2-92ffc0bfd3ff None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:30.984319 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e54dcb-55fc-4a5e-8ad2-92ffc0bfd3ff None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:30.984593 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e54dcb-55fc-4a5e-8ad2-92ffc0bfd3ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:30.985419 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e54dcb-55fc-4a5e-8ad2-92ffc0bfd3ff None admin] RBAC: Adding query filter params (role.id=d1ddee590a964ff1bbb4ee8716173021, scope.project.id=4e8aa015f53442c1add8861248076f46, user.id=ba2177d2a0914eba9e11b306859f6f5f) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:30.985730 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e54dcb-55fc-4a5e-8ad2-92ffc0bfd3ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:30.987697 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e54dcb-55fc-4a5e-8ad2-92ffc0bfd3ff None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:30.994157 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 559/1106] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 22:36:30 2022] GET /identity/v3/role_assignments?user.id=ba2177d2a0914eba9e11b306859f6f5f&scope.project.id=4e8aa015f53442c1add8861248076f46&role.id=d1ddee590a964ff1bbb4ee8716173021 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:33.061760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da79551d-36c1-4c36-997f-cc9c21829568 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:33.062310 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da79551d-36c1-4c36-997f-cc9c21829568 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:33.062310 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da79551d-36c1-4c36-997f-cc9c21829568 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:33.063718 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 548/1107] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:36:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:36:33.068894 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c83b78-3ed0-472d-a51d-d222db0fff8f None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:33.069309 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c83b78-3ed0-472d-a51d-d222db0fff8f None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:33.069446 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c83b78-3ed0-472d-a51d-d222db0fff8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:33.074497 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:33.086812 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-75c83b78-3ed0-472d-a51d-d222db0fff8f None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:33.087990 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-75c83b78-3ed0-472d-a51d-d222db0fff8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:33.153148 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 560/1108] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:36:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:36:33.178571 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da79551d-36c1-4c36-997f-cc9c21829568 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:33.280828 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da79551d-36c1-4c36-997f-cc9c21829568 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:33.294043 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:33.314344 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce02a3-b88e-4538-a56b-d5a770af2c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:33.315662 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce02a3-b88e-4538-a56b-d5a770af2c7f None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:33.315944 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce02a3-b88e-4538-a56b-d5a770af2c7f None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:33.316218 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce02a3-b88e-4538-a56b-d5a770af2c7f None admin] PATH_INFO: `/v3/domains/default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:33.317431 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce02a3-b88e-4538-a56b-d5a770af2c7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:33.319176 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce02a3-b88e-4538-a56b-d5a770af2c7f None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:33.320978 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 549/1109] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:36:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:33.326514 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c83b78-3ed0-472d-a51d-d222db0fff8f None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:33.423484 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c83b78-3ed0-472d-a51d-d222db0fff8f None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:33.452306 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:33.457782 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deae72f-92b0-45b0-908e-5505c522d65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:33.458444 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deae72f-92b0-45b0-908e-5505c522d65f None admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:33.458728 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deae72f-92b0-45b0-908e-5505c522d65f None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:33.459040 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deae72f-92b0-45b0-908e-5505c522d65f None admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:33.461104 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deae72f-92b0-45b0-908e-5505c522d65f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:33.461199 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deae72f-92b0-45b0-908e-5505c522d65f None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:33.576313 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 561/1110] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 21 22:36:33 2022] POST /identity/v3/users => generated 286 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:36:35.649430 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d77f1b-4cf5-4cb6-845e-cd80699b39e6 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:35.649996 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d77f1b-4cf5-4cb6-845e-cd80699b39e6 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:35.649996 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d77f1b-4cf5-4cb6-845e-cd80699b39e6 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:35.651397 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 550/1111] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:36:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:36:35.656581 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc13f7bd-4d6c-4b63-af9b-f5559f12a701 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:35.657027 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc13f7bd-4d6c-4b63-af9b-f5559f12a701 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:35.657130 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc13f7bd-4d6c-4b63-af9b-f5559f12a701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:35.673668 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-dc13f7bd-4d6c-4b63-af9b-f5559f12a701 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:35.674900 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-dc13f7bd-4d6c-4b63-af9b-f5559f12a701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:35.739517 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 562/1112] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:36:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:36:35.762598 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d77f1b-4cf5-4cb6-845e-cd80699b39e6 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:35.860130 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d77f1b-4cf5-4cb6-845e-cd80699b39e6 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:35.892547 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2337273-d5cf-41db-b695-d54755bf2623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:35.893904 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2337273-d5cf-41db-b695-d54755bf2623 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:35.894179 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2337273-d5cf-41db-b695-d54755bf2623 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:35.894446 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2337273-d5cf-41db-b695-d54755bf2623 None admin] PATH_INFO: `/v3/roles/b1849c34fc37410098b94ff687ab0055` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:35.895919 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2337273-d5cf-41db-b695-d54755bf2623 None admin] RBAC: Authorizing `identity:get_role(role_id=b1849c34fc37410098b94ff687ab0055)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:35.897260 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2337273-d5cf-41db-b695-d54755bf2623 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:35.898594 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 551/1113] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 22:36:35 2022] GET /identity/v3/roles/b1849c34fc37410098b94ff687ab0055 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:35.904206 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc13f7bd-4d6c-4b63-af9b-f5559f12a701 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:36.000496 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc13f7bd-4d6c-4b63-af9b-f5559f12a701 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:36.017704 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:36.033892 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411f5e9-cfd4-4683-8430-23783eec3ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:36.037101 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411f5e9-cfd4-4683-8430-23783eec3ca7 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:36.037101 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411f5e9-cfd4-4683-8430-23783eec3ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:36.037101 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411f5e9-cfd4-4683-8430-23783eec3ca7 None admin] PATH_INFO: `/v3/users/63b734cc9325405a9bb9b274c92c7319` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:36.048130 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4411f5e9-cfd4-4683-8430-23783eec3ca7 None admin] RBAC: Authorizing `identity:get_user(user_id=63b734cc9325405a9bb9b274c92c7319)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:36.050174 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4411f5e9-cfd4-4683-8430-23783eec3ca7 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:36.051817 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 563/1114] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 22:36:35 2022] GET /identity/v3/users/63b734cc9325405a9bb9b274c92c7319 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:36.057378 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2337273-d5cf-41db-b695-d54755bf2623 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:36.106632 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:36.162004 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2337273-d5cf-41db-b695-d54755bf2623 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:36.195763 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c72f73-f324-4c7b-845e-7a0c90dbad34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:36.196403 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c72f73-f324-4c7b-845e-7a0c90dbad34 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:36.196688 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c72f73-f324-4c7b-845e-7a0c90dbad34 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:36.196971 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c72f73-f324-4c7b-845e-7a0c90dbad34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:36.262371 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c72f73-f324-4c7b-845e-7a0c90dbad34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:36.264120 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c72f73-f324-4c7b-845e-7a0c90dbad34 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:36.310137 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:36.360337 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 552/1115] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:36:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 22:36:36.366811 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411f5e9-cfd4-4683-8430-23783eec3ca7 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:36.462366 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411f5e9-cfd4-4683-8430-23783eec3ca7 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:36.496228 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0a620f-980a-4af7-8d73-91ab8808c1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:36.497085 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0a620f-980a-4af7-8d73-91ab8808c1b7 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:36.497366 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0a620f-980a-4af7-8d73-91ab8808c1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:36.497639 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0a620f-980a-4af7-8d73-91ab8808c1b7 None admin] PATH_INFO: `/v3/projects/4e8aa015f53442c1add8861248076f46` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:36.498796 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0a620f-980a-4af7-8d73-91ab8808c1b7 None admin] RBAC: Authorizing `identity:get_project(project_id=4e8aa015f53442c1add8861248076f46)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:36.500235 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0a620f-980a-4af7-8d73-91ab8808c1b7 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:36.501913 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 564/1116] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:36:36 2022] GET /identity/v3/projects/4e8aa015f53442c1add8861248076f46 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:36.507976 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c72f73-f324-4c7b-845e-7a0c90dbad34 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:36.605073 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c72f73-f324-4c7b-845e-7a0c90dbad34 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:36.639004 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9990bf8-e937-4d44-8b3c-82f73ab6d43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:36.639650 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9990bf8-e937-4d44-8b3c-82f73ab6d43d None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:36.639934 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9990bf8-e937-4d44-8b3c-82f73ab6d43d None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:36.640211 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9990bf8-e937-4d44-8b3c-82f73ab6d43d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:36.641037 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9990bf8-e937-4d44-8b3c-82f73ab6d43d None admin] RBAC: Adding query filter params (role.id=b1849c34fc37410098b94ff687ab0055, scope.project.id=4e8aa015f53442c1add8861248076f46, user.id=63b734cc9325405a9bb9b274c92c7319) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:36.641351 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9990bf8-e937-4d44-8b3c-82f73ab6d43d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:36.643295 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9990bf8-e937-4d44-8b3c-82f73ab6d43d None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:36.648595 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 553/1117] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 22:36:36 2022] GET /identity/v3/role_assignments?user.id=63b734cc9325405a9bb9b274c92c7319&scope.project.id=4e8aa015f53442c1add8861248076f46&role.id=b1849c34fc37410098b94ff687ab0055 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:38.651138 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e8ae6-4944-46f2-87f4-5d149cba39aa None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:38.651646 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e8ae6-4944-46f2-87f4-5d149cba39aa None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:38.651757 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e8ae6-4944-46f2-87f4-5d149cba39aa None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:38.653036 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 565/1118] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:36:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:36:38.658452 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b10f49-34a8-4c6c-bf37-fb366beb0d57 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:38.658850 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b10f49-34a8-4c6c-bf37-fb366beb0d57 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:38.659048 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b10f49-34a8-4c6c-bf37-fb366beb0d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:38.675530 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-29b10f49-34a8-4c6c-bf37-fb366beb0d57 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:38.676715 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-29b10f49-34a8-4c6c-bf37-fb366beb0d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:38.683935 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:38.742501 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 554/1119] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:36:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:36:38.765917 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e8ae6-4944-46f2-87f4-5d149cba39aa None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:38.811871 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:38.868253 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e8ae6-4944-46f2-87f4-5d149cba39aa None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:38.901273 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5003941f-6f5a-4d48-b1c9-4c5d969982c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:38.902135 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5003941f-6f5a-4d48-b1c9-4c5d969982c6 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:38.902415 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5003941f-6f5a-4d48-b1c9-4c5d969982c6 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:38.902692 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5003941f-6f5a-4d48-b1c9-4c5d969982c6 None admin] PATH_INFO: `/v3/roles/b1849c34fc37410098b94ff687ab0055` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:38.904172 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5003941f-6f5a-4d48-b1c9-4c5d969982c6 None admin] RBAC: Authorizing `identity:get_role(role_id=b1849c34fc37410098b94ff687ab0055)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:38.905514 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5003941f-6f5a-4d48-b1c9-4c5d969982c6 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:38.906869 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 566/1120] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 22:36:38 2022] GET /identity/v3/roles/b1849c34fc37410098b94ff687ab0055 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:38.913353 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b10f49-34a8-4c6c-bf37-fb366beb0d57 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:39.006826 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b10f49-34a8-4c6c-bf37-fb366beb0d57 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:39.041178 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dd23f3-6389-4836-b397-129c5388bf4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:39.041966 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd23f3-6389-4836-b397-129c5388bf4f None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:39.042241 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd23f3-6389-4836-b397-129c5388bf4f None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:39.042509 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd23f3-6389-4836-b397-129c5388bf4f None admin] PATH_INFO: `/v3/users/63b734cc9325405a9bb9b274c92c7319` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:39.043693 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dd23f3-6389-4836-b397-129c5388bf4f None admin] RBAC: Authorizing `identity:get_user(user_id=63b734cc9325405a9bb9b274c92c7319)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:39.045093 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dd23f3-6389-4836-b397-129c5388bf4f None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:39.046667 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 555/1121] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 22:36:38 2022] GET /identity/v3/users/63b734cc9325405a9bb9b274c92c7319 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:39.052222 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5003941f-6f5a-4d48-b1c9-4c5d969982c6 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:39.148503 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5003941f-6f5a-4d48-b1c9-4c5d969982c6 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:39.161625 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:39.181432 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ddc70-5150-47b0-8194-b151d7dc3c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:39.182274 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ddc70-5150-47b0-8194-b151d7dc3c17 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:39.182555 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ddc70-5150-47b0-8194-b151d7dc3c17 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:39.182864 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ddc70-5150-47b0-8194-b151d7dc3c17 None admin] PATH_INFO: `/v3/projects/4e8aa015f53442c1add8861248076f46` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:39.184027 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84ddc70-5150-47b0-8194-b151d7dc3c17 None admin] RBAC: Authorizing `identity:get_project(project_id=4e8aa015f53442c1add8861248076f46)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:39.185436 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84ddc70-5150-47b0-8194-b151d7dc3c17 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:39.187194 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 567/1122] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:36:39 2022] GET /identity/v3/projects/4e8aa015f53442c1add8861248076f46 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:39.193011 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dd23f3-6389-4836-b397-129c5388bf4f None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:39.205627 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:39.290975 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dd23f3-6389-4836-b397-129c5388bf4f None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:39.324090 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7931e1ca-6107-41c9-9efd-ad466b0ec851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:39.324901 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7931e1ca-6107-41c9-9efd-ad466b0ec851 None admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:39.325181 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7931e1ca-6107-41c9-9efd-ad466b0ec851 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:39.325455 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7931e1ca-6107-41c9-9efd-ad466b0ec851 None admin] PATH_INFO: `/v3/projects/4e8aa015f53442c1add8861248076f46/users/63b734cc9325405a9bb9b274c92c7319/roles/b1849c34fc37410098b94ff687ab0055` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:39.327641 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7931e1ca-6107-41c9-9efd-ad466b0ec851 None admin] RBAC: Authorizing `identity:create_grant(project_id=4e8aa015f53442c1add8861248076f46, user_id=63b734cc9325405a9bb9b274c92c7319, role_id=b1849c34fc37410098b94ff687ab0055)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:39.329047 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7931e1ca-6107-41c9-9efd-ad466b0ec851 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:39.405315 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 556/1123] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 21 22:36:39 2022] PUT /identity/v3/projects/4e8aa015f53442c1add8861248076f46/users/63b734cc9325405a9bb9b274c92c7319/roles/b1849c34fc37410098b94ff687ab0055 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:36:41.426965 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8441ebc2-a192-445b-a5ac-e3ccd770b56b None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:41.427516 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8441ebc2-a192-445b-a5ac-e3ccd770b56b None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:41.427516 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8441ebc2-a192-445b-a5ac-e3ccd770b56b None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:41.428855 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 568/1124] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:36:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:36:41.433936 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e4632-9000-4b0a-9d37-0c2518769b7f None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:41.434327 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e4632-9000-4b0a-9d37-0c2518769b7f None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:41.434489 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e4632-9000-4b0a-9d37-0c2518769b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:41.450714 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-1b2e4632-9000-4b0a-9d37-0c2518769b7f None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:41.451916 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-1b2e4632-9000-4b0a-9d37-0c2518769b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:41.515394 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:41.520465 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 557/1125] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:36:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:36:41.543788 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8441ebc2-a192-445b-a5ac-e3ccd770b56b None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:41.644382 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8441ebc2-a192-445b-a5ac-e3ccd770b56b None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:41.676755 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b98c3ca-6e19-4fbb-88fb-07531481ad8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:41.677673 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98c3ca-6e19-4fbb-88fb-07531481ad8e None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:41.677895 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98c3ca-6e19-4fbb-88fb-07531481ad8e None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:41.678170 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98c3ca-6e19-4fbb-88fb-07531481ad8e None admin] PATH_INFO: `/v3/roles/b1849c34fc37410098b94ff687ab0055` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:41.679677 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b98c3ca-6e19-4fbb-88fb-07531481ad8e None admin] RBAC: Authorizing `identity:get_role(role_id=b1849c34fc37410098b94ff687ab0055)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:41.681066 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b98c3ca-6e19-4fbb-88fb-07531481ad8e None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:41.682424 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 569/1126] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 22:36:41 2022] GET /identity/v3/roles/b1849c34fc37410098b94ff687ab0055 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:41.688124 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2e4632-9000-4b0a-9d37-0c2518769b7f None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:41.784792 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2e4632-9000-4b0a-9d37-0c2518769b7f None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:41.817538 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a87959-27f9-4b9d-bd59-575aedb0a71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:41.818393 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a87959-27f9-4b9d-bd59-575aedb0a71f None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:41.818677 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a87959-27f9-4b9d-bd59-575aedb0a71f None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:41.818986 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a87959-27f9-4b9d-bd59-575aedb0a71f None admin] PATH_INFO: `/v3/users/63b734cc9325405a9bb9b274c92c7319` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:41.820294 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a87959-27f9-4b9d-bd59-575aedb0a71f None admin] RBAC: Authorizing `identity:get_user(user_id=63b734cc9325405a9bb9b274c92c7319)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:41.821711 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a87959-27f9-4b9d-bd59-575aedb0a71f None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:41.823435 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 558/1127] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 22:36:41 2022] GET /identity/v3/users/63b734cc9325405a9bb9b274c92c7319 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:41.828988 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b98c3ca-6e19-4fbb-88fb-07531481ad8e None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:41.878262 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:41.928506 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b98c3ca-6e19-4fbb-88fb-07531481ad8e None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:41.961503 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db658391-137a-46db-b44f-299e28a8af01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:41.962140 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db658391-137a-46db-b44f-299e28a8af01 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:41.962422 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db658391-137a-46db-b44f-299e28a8af01 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:41.962697 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db658391-137a-46db-b44f-299e28a8af01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:42.027716 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db658391-137a-46db-b44f-299e28a8af01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:42.029140 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db658391-137a-46db-b44f-299e28a8af01 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:42.074965 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:42.124537 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 570/1128] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:36:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 22:36:42.130977 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a87959-27f9-4b9d-bd59-575aedb0a71f None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:42.195890 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:42.228025 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a87959-27f9-4b9d-bd59-575aedb0a71f None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:42.261239 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b76be3-0baa-41a1-be0e-3c19ba486d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:42.262082 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b76be3-0baa-41a1-be0e-3c19ba486d75 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:42.262363 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b76be3-0baa-41a1-be0e-3c19ba486d75 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:42.262636 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b76be3-0baa-41a1-be0e-3c19ba486d75 None admin] PATH_INFO: `/v3/projects/4e8aa015f53442c1add8861248076f46` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:42.263818 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b76be3-0baa-41a1-be0e-3c19ba486d75 None admin] RBAC: Authorizing `identity:get_project(project_id=4e8aa015f53442c1add8861248076f46)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:42.265228 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b76be3-0baa-41a1-be0e-3c19ba486d75 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:42.266910 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 559/1129] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:36:42 2022] GET /identity/v3/projects/4e8aa015f53442c1add8861248076f46 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:42.273224 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db658391-137a-46db-b44f-299e28a8af01 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:42.368620 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db658391-137a-46db-b44f-299e28a8af01 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:42.401479 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b17822-8068-4a20-b90e-df3fb9a6b3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:42.402106 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b17822-8068-4a20-b90e-df3fb9a6b3f2 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:42.402383 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b17822-8068-4a20-b90e-df3fb9a6b3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:42.402652 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b17822-8068-4a20-b90e-df3fb9a6b3f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:42.403496 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b17822-8068-4a20-b90e-df3fb9a6b3f2 None admin] RBAC: Adding query filter params (role.id=b1849c34fc37410098b94ff687ab0055, scope.project.id=4e8aa015f53442c1add8861248076f46, user.id=63b734cc9325405a9bb9b274c92c7319) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:42.403808 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b17822-8068-4a20-b90e-df3fb9a6b3f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:42.405205 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b17822-8068-4a20-b90e-df3fb9a6b3f2 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:42.410817 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 571/1130] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 22:36:42 2022] GET /identity/v3/role_assignments?user.id=63b734cc9325405a9bb9b274c92c7319&scope.project.id=4e8aa015f53442c1add8861248076f46&role.id=b1849c34fc37410098b94ff687ab0055 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:44.458537 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0ae3f-38d7-4029-923c-99b9ddd34edf None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:44.459091 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0ae3f-38d7-4029-923c-99b9ddd34edf None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:44.459200 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0ae3f-38d7-4029-923c-99b9ddd34edf None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:44.460505 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 560/1131] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:36:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:36:44.465777 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3bb9f5-406d-4cdd-8598-486e54dd8f64 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:44.466204 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3bb9f5-406d-4cdd-8598-486e54dd8f64 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:44.466330 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3bb9f5-406d-4cdd-8598-486e54dd8f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:44.486698 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-7a3bb9f5-406d-4cdd-8598-486e54dd8f64 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:44.487894 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-7a3bb9f5-406d-4cdd-8598-486e54dd8f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:44.495211 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:44.553255 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 572/1132] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:36:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:36:44.577319 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d0ae3f-38d7-4029-923c-99b9ddd34edf None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:44.685125 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d0ae3f-38d7-4029-923c-99b9ddd34edf None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:44.716939 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62b0b76-25b4-42c0-8d9d-2e67a268fd5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:44.717660 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62b0b76-25b4-42c0-8d9d-2e67a268fd5a None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:44.717939 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62b0b76-25b4-42c0-8d9d-2e67a268fd5a None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:44.718209 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62b0b76-25b4-42c0-8d9d-2e67a268fd5a None admin] PATH_INFO: `/v3/domains/default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:44.719372 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62b0b76-25b4-42c0-8d9d-2e67a268fd5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:44.721315 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62b0b76-25b4-42c0-8d9d-2e67a268fd5a None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:44.722929 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 561/1133] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:36:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:44.729156 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3bb9f5-406d-4cdd-8598-486e54dd8f64 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:44.825555 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3bb9f5-406d-4cdd-8598-486e54dd8f64 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:44.859241 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b53d33-1410-4c2b-ac68-bbfc0523057d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:44.859895 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b53d33-1410-4c2b-ac68-bbfc0523057d None admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:44.860180 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b53d33-1410-4c2b-ac68-bbfc0523057d None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:44.860456 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b53d33-1410-4c2b-ac68-bbfc0523057d None admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:44.862751 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b53d33-1410-4c2b-ac68-bbfc0523057d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:44.864205 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b53d33-1410-4c2b-ac68-bbfc0523057d None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:44.876798 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:44.953233 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 573/1134] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jul 21 22:36:44 2022] POST /identity/v3/projects => generated 311 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:36:46.990858 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f815c-0d6e-4dad-94c1-2c5db237bdb5 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:46.991413 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f815c-0d6e-4dad-94c1-2c5db237bdb5 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:46.991413 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f815c-0d6e-4dad-94c1-2c5db237bdb5 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:46.992783 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 562/1135] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:36:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:36:46.998082 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668bb377-7667-4ebf-a627-955eccb68903 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:46.998496 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668bb377-7667-4ebf-a627-955eccb68903 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:46.998638 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668bb377-7667-4ebf-a627-955eccb68903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:47.015425 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-668bb377-7667-4ebf-a627-955eccb68903 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:47.016588 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-668bb377-7667-4ebf-a627-955eccb68903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:47.081631 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 574/1136] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:36:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:36:47.107155 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55f815c-0d6e-4dad-94c1-2c5db237bdb5 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:47.110914 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:47.206043 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55f815c-0d6e-4dad-94c1-2c5db237bdb5 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:47.239496 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78a1d02-6701-47f0-8822-f4b4f2374dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:47.240269 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78a1d02-6701-47f0-8822-f4b4f2374dea None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:47.240550 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78a1d02-6701-47f0-8822-f4b4f2374dea None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:47.240825 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78a1d02-6701-47f0-8822-f4b4f2374dea None admin] PATH_INFO: `/v3/domains/default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:47.242028 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78a1d02-6701-47f0-8822-f4b4f2374dea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:47.243418 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78a1d02-6701-47f0-8822-f4b4f2374dea None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:47.245175 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 563/1137] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:36:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:47.250684 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668bb377-7667-4ebf-a627-955eccb68903 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:47.346922 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668bb377-7667-4ebf-a627-955eccb68903 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:47.369592 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:47.380157 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078025ca-a03a-441b-b592-8eedb140610c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:47.380810 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078025ca-a03a-441b-b592-8eedb140610c None admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:47.381094 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078025ca-a03a-441b-b592-8eedb140610c None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:47.381368 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078025ca-a03a-441b-b592-8eedb140610c None admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:47.382067 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078025ca-a03a-441b-b592-8eedb140610c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:47.383508 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078025ca-a03a-441b-b592-8eedb140610c None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:47.502829 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 575/1138] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 21 22:36:47 2022] POST /identity/v3/users => generated 286 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:36:49.575715 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4847d121-2147-427b-bc3d-f5c7182dd3a9 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:49.576165 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4847d121-2147-427b-bc3d-f5c7182dd3a9 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:49.576249 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4847d121-2147-427b-bc3d-f5c7182dd3a9 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:49.577662 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 564/1139] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:36:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:36:49.582909 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b13ce08-3c8c-4f3f-8530-c23178df9fe3 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:49.583334 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b13ce08-3c8c-4f3f-8530-c23178df9fe3 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:49.583472 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b13ce08-3c8c-4f3f-8530-c23178df9fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:49.600017 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-1b13ce08-3c8c-4f3f-8530-c23178df9fe3 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:49.601195 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-1b13ce08-3c8c-4f3f-8530-c23178df9fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:49.665924 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 576/1140] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:36:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:36:49.689335 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4847d121-2147-427b-bc3d-f5c7182dd3a9 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:49.760933 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:49.791240 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4847d121-2147-427b-bc3d-f5c7182dd3a9 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:49.823865 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e26d63-e8b7-477e-88ca-17ddff85695f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:49.824636 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e26d63-e8b7-477e-88ca-17ddff85695f None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:49.824947 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e26d63-e8b7-477e-88ca-17ddff85695f None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:49.825222 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e26d63-e8b7-477e-88ca-17ddff85695f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:49.834969 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e26d63-e8b7-477e-88ca-17ddff85695f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:36:49.842356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-a4e26d63-e8b7-477e-88ca-17ddff85695f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:36:49.842356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:49.842356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:49.842356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:49.842356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:49.842356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:49.842356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:49.842356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:49.842356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:49.842356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:49.842356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:49.842356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:49.842356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:36:49.842356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:36:49.842356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:36:49.842356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:36:49.843540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:36:49.843540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:36:49.843540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:36:49.843540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:36:49.843540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:49.843540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:49.843540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:36:49.843540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:36:49.843540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:36:49.843540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:36:49.843540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:36:49.843540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:36:49.843540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:36:49.843540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:36:49.843540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:36:49.843540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:36:49.844625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:36:49.844625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:36:49.844625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:36:49.844625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:36:49.844625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:36:49.844625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:36:49.844625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:36:49.844625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:36:49.844625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:36:49.844625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:36:49.844625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:36:49.844625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:36:49.844625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 565/1141] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 22:36:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:36:49.850276 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b13ce08-3c8c-4f3f-8530-c23178df9fe3 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:49.947680 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b13ce08-3c8c-4f3f-8530-c23178df9fe3 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:49.957960 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:49.981266 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449e1c2a-e6d0-4917-bed7-5e27eb9ec10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:49.981924 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449e1c2a-e6d0-4917-bed7-5e27eb9ec10d None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:49.982210 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449e1c2a-e6d0-4917-bed7-5e27eb9ec10d None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:49.982487 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449e1c2a-e6d0-4917-bed7-5e27eb9ec10d None admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:49.983262 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449e1c2a-e6d0-4917-bed7-5e27eb9ec10d None admin] RBAC: Adding query filter params (name=admin) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:49.983573 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449e1c2a-e6d0-4917-bed7-5e27eb9ec10d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:49.984989 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449e1c2a-e6d0-4917-bed7-5e27eb9ec10d None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:49.993870 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 577/1142] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:36:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:49.999524 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e26d63-e8b7-477e-88ca-17ddff85695f None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:50.095421 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e26d63-e8b7-477e-88ca-17ddff85695f None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:50.124943 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:50.129695 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d05b22-1832-4ebe-b0f7-c3ce392ef79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:50.134514 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d05b22-1832-4ebe-b0f7-c3ce392ef79c None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:50.134514 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d05b22-1832-4ebe-b0f7-c3ce392ef79c None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:50.134514 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d05b22-1832-4ebe-b0f7-c3ce392ef79c None admin] PATH_INFO: `/v3/users/f6982fb976d4454f9e770be5bd8bbe60` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:50.144081 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d05b22-1832-4ebe-b0f7-c3ce392ef79c None admin] RBAC: Authorizing `identity:get_user(user_id=f6982fb976d4454f9e770be5bd8bbe60)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:50.145520 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d05b22-1832-4ebe-b0f7-c3ce392ef79c None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:50.147162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 566/1143] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 22:36:49 2022] GET /identity/v3/users/f6982fb976d4454f9e770be5bd8bbe60 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:50.152842 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449e1c2a-e6d0-4917-bed7-5e27eb9ec10d None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:50.249850 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449e1c2a-e6d0-4917-bed7-5e27eb9ec10d None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:50.281704 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd35540-513c-4db7-bc1e-1b4e07f85449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:50.282333 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd35540-513c-4db7-bc1e-1b4e07f85449 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:50.282643 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd35540-513c-4db7-bc1e-1b4e07f85449 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:50.282945 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd35540-513c-4db7-bc1e-1b4e07f85449 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:50.321578 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:50.347422 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd35540-513c-4db7-bc1e-1b4e07f85449 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:50.348782 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd35540-513c-4db7-bc1e-1b4e07f85449 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:50.442225 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 578/1144] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:36:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 22:36:50.447978 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d05b22-1832-4ebe-b0f7-c3ce392ef79c None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:50.546405 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d05b22-1832-4ebe-b0f7-c3ce392ef79c None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:50.579595 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8697e272-0dc6-41e6-89c8-8e0bdc14a0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:50.581066 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8697e272-0dc6-41e6-89c8-8e0bdc14a0aa None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:50.581351 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8697e272-0dc6-41e6-89c8-8e0bdc14a0aa None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:50.581624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8697e272-0dc6-41e6-89c8-8e0bdc14a0aa None admin] PATH_INFO: `/v3/projects/fda9acee8e324474b4f50dbdc277d2b2` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:50.582814 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8697e272-0dc6-41e6-89c8-8e0bdc14a0aa None admin] RBAC: Authorizing `identity:get_project(project_id=fda9acee8e324474b4f50dbdc277d2b2)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:50.584238 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8697e272-0dc6-41e6-89c8-8e0bdc14a0aa None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:50.585932 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 567/1145] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:36:50 2022] GET /identity/v3/projects/fda9acee8e324474b4f50dbdc277d2b2 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:50.591951 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd35540-513c-4db7-bc1e-1b4e07f85449 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:50.660333 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:50.687734 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd35540-513c-4db7-bc1e-1b4e07f85449 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:50.721364 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b6784a-87f3-4a59-9624-835e9b96fe2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:50.722003 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6784a-87f3-4a59-9624-835e9b96fe2d None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:50.722282 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6784a-87f3-4a59-9624-835e9b96fe2d None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:50.722552 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6784a-87f3-4a59-9624-835e9b96fe2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:50.723403 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b6784a-87f3-4a59-9624-835e9b96fe2d None admin] RBAC: Adding query filter params (role.id=d1ddee590a964ff1bbb4ee8716173021, scope.project.id=fda9acee8e324474b4f50dbdc277d2b2, user.id=f6982fb976d4454f9e770be5bd8bbe60) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:50.723714 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b6784a-87f3-4a59-9624-835e9b96fe2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:50.725392 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b6784a-87f3-4a59-9624-835e9b96fe2d None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:50.730668 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 579/1146] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 22:36:50 2022] GET /identity/v3/role_assignments?user.id=f6982fb976d4454f9e770be5bd8bbe60&scope.project.id=fda9acee8e324474b4f50dbdc277d2b2&role.id=d1ddee590a964ff1bbb4ee8716173021 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:52.732100 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2ef9c7-ac4c-4f3f-80c6-051b67feb6c4 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:52.732614 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2ef9c7-ac4c-4f3f-80c6-051b67feb6c4 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:52.732723 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2ef9c7-ac4c-4f3f-80c6-051b67feb6c4 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:52.734034 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 568/1147] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:36:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:36:52.739384 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee5463-4c01-43ef-8779-a0f301fc3ff0 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:52.739802 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee5463-4c01-43ef-8779-a0f301fc3ff0 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:52.739935 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee5463-4c01-43ef-8779-a0f301fc3ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:52.756836 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-4fee5463-4c01-43ef-8779-a0f301fc3ff0 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:52.757950 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-4fee5463-4c01-43ef-8779-a0f301fc3ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:52.823128 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 580/1148] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:36:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:36:52.845705 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2ef9c7-ac4c-4f3f-80c6-051b67feb6c4 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:52.904409 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:52.944965 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2ef9c7-ac4c-4f3f-80c6-051b67feb6c4 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:52.977604 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d467b66e-10ea-4eab-8f17-096d92dc28f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:52.978413 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467b66e-10ea-4eab-8f17-096d92dc28f2 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:52.978701 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467b66e-10ea-4eab-8f17-096d92dc28f2 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:52.979009 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467b66e-10ea-4eab-8f17-096d92dc28f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:52.988961 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d467b66e-10ea-4eab-8f17-096d92dc28f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:36:52.996511 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-d467b66e-10ea-4eab-8f17-096d92dc28f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:36:52.996511 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:52.996511 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:52.996511 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:52.996511 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:52.996511 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:52.996511 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:52.996511 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:52.996511 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:52.996511 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:52.996511 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:52.996511 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:52.996511 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:36:52.996511 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:36:52.996511 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:36:52.996511 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:36:52.997575 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:36:52.997575 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:36:52.997575 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:36:52.997575 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:36:52.997575 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:52.997575 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:52.997575 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:36:52.997575 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:36:52.997575 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:36:52.997575 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:36:52.997575 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:36:52.997575 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:36:52.997575 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:36:52.997575 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:36:52.997575 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:36:52.997575 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:36:52.998707 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:36:52.998707 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:36:52.998707 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:36:52.998707 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:36:52.998707 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:36:52.998707 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:36:52.998707 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:36:52.998707 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:36:52.998707 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:36:52.998707 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:36:52.998707 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:36:52.998707 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:36:52.998707 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 569/1149] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 22:36:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:36:53.004228 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee5463-4c01-43ef-8779-a0f301fc3ff0 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:53.067061 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:53.099820 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee5463-4c01-43ef-8779-a0f301fc3ff0 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:53.134573 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbfb4cd-8d4d-4d61-a024-f5268242ea01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:53.135270 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbfb4cd-8d4d-4d61-a024-f5268242ea01 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:53.135550 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbfb4cd-8d4d-4d61-a024-f5268242ea01 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:53.135825 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbfb4cd-8d4d-4d61-a024-f5268242ea01 None admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:53.136564 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbfb4cd-8d4d-4d61-a024-f5268242ea01 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:53.136867 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbfb4cd-8d4d-4d61-a024-f5268242ea01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:53.138218 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbfb4cd-8d4d-4d61-a024-f5268242ea01 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:53.146104 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 581/1150] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:36:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:53.151841 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d467b66e-10ea-4eab-8f17-096d92dc28f2 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:53.249030 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d467b66e-10ea-4eab-8f17-096d92dc28f2 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:53.265772 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:53.282008 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb3d1e-c211-465f-bac5-4062322a5414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:53.282890 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb3d1e-c211-465f-bac5-4062322a5414 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:53.283231 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb3d1e-c211-465f-bac5-4062322a5414 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:53.283552 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb3d1e-c211-465f-bac5-4062322a5414 None admin] PATH_INFO: `/v3/users/f6982fb976d4454f9e770be5bd8bbe60` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:53.284791 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beb3d1e-c211-465f-bac5-4062322a5414 None admin] RBAC: Authorizing `identity:get_user(user_id=f6982fb976d4454f9e770be5bd8bbe60)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:53.286241 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beb3d1e-c211-465f-bac5-4062322a5414 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:53.287924 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 570/1151] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 22:36:53 2022] GET /identity/v3/users/f6982fb976d4454f9e770be5bd8bbe60 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:53.293597 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbfb4cd-8d4d-4d61-a024-f5268242ea01 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:53.394712 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbfb4cd-8d4d-4d61-a024-f5268242ea01 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:53.419589 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:53.427596 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3fd026-5e24-4e20-9ac6-32e44738f932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:53.428447 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3fd026-5e24-4e20-9ac6-32e44738f932 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:53.430528 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3fd026-5e24-4e20-9ac6-32e44738f932 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:53.430528 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3fd026-5e24-4e20-9ac6-32e44738f932 None admin] PATH_INFO: `/v3/projects/fda9acee8e324474b4f50dbdc277d2b2` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:53.430528 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3fd026-5e24-4e20-9ac6-32e44738f932 None admin] RBAC: Authorizing `identity:get_project(project_id=fda9acee8e324474b4f50dbdc277d2b2)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:53.431559 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3fd026-5e24-4e20-9ac6-32e44738f932 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:53.433234 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 582/1152] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:36:53 2022] GET /identity/v3/projects/fda9acee8e324474b4f50dbdc277d2b2 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:53.439870 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb3d1e-c211-465f-bac5-4062322a5414 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:53.537697 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb3d1e-c211-465f-bac5-4062322a5414 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:53.570254 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157bc9f9-5f27-454b-abcf-b8f46f32c823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:53.571168 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157bc9f9-5f27-454b-abcf-b8f46f32c823 None admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:53.571452 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157bc9f9-5f27-454b-abcf-b8f46f32c823 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:53.571726 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157bc9f9-5f27-454b-abcf-b8f46f32c823 None admin] PATH_INFO: `/v3/projects/fda9acee8e324474b4f50dbdc277d2b2/users/f6982fb976d4454f9e770be5bd8bbe60/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:53.573672 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157bc9f9-5f27-454b-abcf-b8f46f32c823 None admin] RBAC: Authorizing `identity:create_grant(project_id=fda9acee8e324474b4f50dbdc277d2b2, user_id=f6982fb976d4454f9e770be5bd8bbe60, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:53.575114 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157bc9f9-5f27-454b-abcf-b8f46f32c823 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:53.660564 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 571/1153] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 21 22:36:53 2022] PUT /identity/v3/projects/fda9acee8e324474b4f50dbdc277d2b2/users/f6982fb976d4454f9e770be5bd8bbe60/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:36:55.668167 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a77ff-42de-4aea-a342-008ffa366150 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:55.668680 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a77ff-42de-4aea-a342-008ffa366150 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:55.668764 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a77ff-42de-4aea-a342-008ffa366150 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:55.670064 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 583/1154] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:36:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:36:55.676655 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48449a-b5c3-4274-ab17-75323c7fd8f5 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:55.677051 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48449a-b5c3-4274-ab17-75323c7fd8f5 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:55.677208 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48449a-b5c3-4274-ab17-75323c7fd8f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:55.694251 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-ba48449a-b5c3-4274-ab17-75323c7fd8f5 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:55.695470 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-ba48449a-b5c3-4274-ab17-75323c7fd8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:55.736518 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:55.760739 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 572/1155] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:36:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:36:55.784324 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36a77ff-42de-4aea-a342-008ffa366150 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:55.884228 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36a77ff-42de-4aea-a342-008ffa366150 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:55.916819 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43a2c6c-e5b1-43fc-b7fd-5d9baee3e9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:55.917607 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43a2c6c-e5b1-43fc-b7fd-5d9baee3e9b8 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:55.917894 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43a2c6c-e5b1-43fc-b7fd-5d9baee3e9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:55.918173 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43a2c6c-e5b1-43fc-b7fd-5d9baee3e9b8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:55.929379 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43a2c6c-e5b1-43fc-b7fd-5d9baee3e9b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:36:55.936954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-b43a2c6c-e5b1-43fc-b7fd-5d9baee3e9b8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:36:55.936954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:55.936954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:55.936954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:55.936954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:55.936954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:55.936954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:55.936954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:55.936954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:55.936954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:55.936954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:55.936954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:55.936954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:36:55.936954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:36:55.936954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:36:55.936954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:36:55.938084 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:36:55.938084 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:36:55.938084 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:36:55.938084 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:36:55.938084 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:55.938084 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:55.938084 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:36:55.938084 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:36:55.938084 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:36:55.938084 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:36:55.938084 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:36:55.938084 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:36:55.938084 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:36:55.938084 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:36:55.938084 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:36:55.938084 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:36:55.939523 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:36:55.939523 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:36:55.939523 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:36:55.939523 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:36:55.939523 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:36:55.939523 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:36:55.939523 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:36:55.939523 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:36:55.939523 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:36:55.939523 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:36:55.939523 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:36:55.939523 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:36:55.939523 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 584/1156] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 22:36:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 22:36:55.944538 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba48449a-b5c3-4274-ab17-75323c7fd8f5 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:56.049086 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba48449a-b5c3-4274-ab17-75323c7fd8f5 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:56.081463 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb7d45-d5e5-450d-8a6b-f0f2e7cc1df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:56.082125 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb7d45-d5e5-450d-8a6b-f0f2e7cc1df4 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:56.082411 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb7d45-d5e5-450d-8a6b-f0f2e7cc1df4 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:56.082688 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb7d45-d5e5-450d-8a6b-f0f2e7cc1df4 None admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:56.083465 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afb7d45-d5e5-450d-8a6b-f0f2e7cc1df4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:56.083773 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afb7d45-d5e5-450d-8a6b-f0f2e7cc1df4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:56.085130 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afb7d45-d5e5-450d-8a6b-f0f2e7cc1df4 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:56.093069 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 573/1157] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:36:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:56.099730 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43a2c6c-e5b1-43fc-b7fd-5d9baee3e9b8 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:56.154317 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:56.198026 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43a2c6c-e5b1-43fc-b7fd-5d9baee3e9b8 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:56.232755 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a567e21-d26b-45ab-b6d1-6c666c8a9a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:56.233605 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a567e21-d26b-45ab-b6d1-6c666c8a9a62 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:56.233888 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a567e21-d26b-45ab-b6d1-6c666c8a9a62 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:56.234164 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a567e21-d26b-45ab-b6d1-6c666c8a9a62 None admin] PATH_INFO: `/v3/users/f6982fb976d4454f9e770be5bd8bbe60` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:56.235364 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a567e21-d26b-45ab-b6d1-6c666c8a9a62 None admin] RBAC: Authorizing `identity:get_user(user_id=f6982fb976d4454f9e770be5bd8bbe60)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:56.236835 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a567e21-d26b-45ab-b6d1-6c666c8a9a62 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:56.238432 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 585/1158] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 22:36:56 2022] GET /identity/v3/users/f6982fb976d4454f9e770be5bd8bbe60 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:56.244216 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb7d45-d5e5-450d-8a6b-f0f2e7cc1df4 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:56.284320 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:56.343303 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb7d45-d5e5-450d-8a6b-f0f2e7cc1df4 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:56.376638 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f90697-6803-4a7b-8469-20d8c192169d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:56.377278 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f90697-6803-4a7b-8469-20d8c192169d None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:56.377561 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f90697-6803-4a7b-8469-20d8c192169d None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:56.377840 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f90697-6803-4a7b-8469-20d8c192169d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:56.442366 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f90697-6803-4a7b-8469-20d8c192169d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:56.443823 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f90697-6803-4a7b-8469-20d8c192169d None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:56.481028 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:56.541173 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 574/1159] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:36:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 22:36:56.547132 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a567e21-d26b-45ab-b6d1-6c666c8a9a62 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:56.646157 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a567e21-d26b-45ab-b6d1-6c666c8a9a62 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:56.668564 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:56.678874 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02606dbf-0c88-4da9-bef0-2ff7a62fa92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:56.679720 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02606dbf-0c88-4da9-bef0-2ff7a62fa92a None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:56.680006 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02606dbf-0c88-4da9-bef0-2ff7a62fa92a None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:56.680283 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02606dbf-0c88-4da9-bef0-2ff7a62fa92a None admin] PATH_INFO: `/v3/projects/fda9acee8e324474b4f50dbdc277d2b2` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:56.681461 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02606dbf-0c88-4da9-bef0-2ff7a62fa92a None admin] RBAC: Authorizing `identity:get_project(project_id=fda9acee8e324474b4f50dbdc277d2b2)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:56.682974 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02606dbf-0c88-4da9-bef0-2ff7a62fa92a None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:56.684653 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 586/1160] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:36:56 2022] GET /identity/v3/projects/fda9acee8e324474b4f50dbdc277d2b2 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:56.691459 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f90697-6803-4a7b-8469-20d8c192169d None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:56.788168 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f90697-6803-4a7b-8469-20d8c192169d None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:56.820532 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809294a4-e373-40c3-9f2e-f75f14507af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:56.821169 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809294a4-e373-40c3-9f2e-f75f14507af0 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:56.821449 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809294a4-e373-40c3-9f2e-f75f14507af0 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:56.821721 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809294a4-e373-40c3-9f2e-f75f14507af0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:56.822537 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809294a4-e373-40c3-9f2e-f75f14507af0 None admin] RBAC: Adding query filter params (role.id=d1ddee590a964ff1bbb4ee8716173021, scope.project.id=fda9acee8e324474b4f50dbdc277d2b2, user.id=f6982fb976d4454f9e770be5bd8bbe60) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:56.822878 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809294a4-e373-40c3-9f2e-f75f14507af0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:56.824291 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809294a4-e373-40c3-9f2e-f75f14507af0 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:56.830499 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 575/1161] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 22:36:56 2022] GET /identity/v3/role_assignments?user.id=f6982fb976d4454f9e770be5bd8bbe60&scope.project.id=fda9acee8e324474b4f50dbdc277d2b2&role.id=d1ddee590a964ff1bbb4ee8716173021 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:36:58.870363 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b7c64-6060-448c-b318-2ba0e069a83d None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:58.870881 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b7c64-6060-448c-b318-2ba0e069a83d None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:58.870973 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b7c64-6060-448c-b318-2ba0e069a83d None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:58.872318 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 587/1162] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:36:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:36:58.877726 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e043228d-f93e-4ae7-9af5-aa04eaa6d5c8 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:58.878472 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e043228d-f93e-4ae7-9af5-aa04eaa6d5c8 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:58.878752 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e043228d-f93e-4ae7-9af5-aa04eaa6d5c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:58.883711 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:58.895392 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-e043228d-f93e-4ae7-9af5-aa04eaa6d5c8 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:36:58.902975 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-e043228d-f93e-4ae7-9af5-aa04eaa6d5c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:36:58.960357 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 576/1163] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:36:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:36:58.982955 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b7c64-6060-448c-b318-2ba0e069a83d None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:59.081131 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b7c64-6060-448c-b318-2ba0e069a83d None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:59.114952 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9016ce8e-2284-4a09-81a3-bf9c1abffa3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:59.115585 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9016ce8e-2284-4a09-81a3-bf9c1abffa3f None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:59.115866 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9016ce8e-2284-4a09-81a3-bf9c1abffa3f None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:59.116141 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9016ce8e-2284-4a09-81a3-bf9c1abffa3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:59.172190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:59.182027 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9016ce8e-2284-4a09-81a3-bf9c1abffa3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:59.185705 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9016ce8e-2284-4a09-81a3-bf9c1abffa3f None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:59.284468 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 588/1164] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:36:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 22:36:59.290368 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e043228d-f93e-4ae7-9af5-aa04eaa6d5c8 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:59.384402 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e043228d-f93e-4ae7-9af5-aa04eaa6d5c8 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:59.412766 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:59.417368 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f6fab8-c3d5-4407-b579-feddd3b9cd16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:59.418151 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f6fab8-c3d5-4407-b579-feddd3b9cd16 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:59.418431 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f6fab8-c3d5-4407-b579-feddd3b9cd16 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:59.418701 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f6fab8-c3d5-4407-b579-feddd3b9cd16 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:59.425812 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f6fab8-c3d5-4407-b579-feddd3b9cd16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:59.426620 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f6fab8-c3d5-4407-b579-feddd3b9cd16 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:59.440681 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-e2f6fab8-c3d5-4407-b579-feddd3b9cd16 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 21 22:36:59.440681 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:59.440681 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 21 22:36:59.440681 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 21 22:36:59.440681 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:36:59.440681 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:36:59.440681 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:36:59.440681 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:36:59.440681 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 21 22:36:59.440681 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:36:59.440681 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:36:59.440681 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:36:59.440681 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:36:59.440681 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:36:59.440681 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:36:59.440681 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:36:59.440681 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:36:59.441881 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:36:59.441881 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:36:59.441881 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:36:59.441881 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:36:59.441881 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:36:59.441881 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:36:59.441881 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:36:59.441881 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:36:59.441881 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:36:59.441881 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:36:59.441881 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:36:59.441881 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:36:59.441881 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:36:59.441881 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:36:59.441881 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:36:59.441881 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:36:59.443019 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:36:59.443019 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:36:59.443019 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:36:59.443019 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:36:59.443019 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:36:59.443019 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:36:59.443019 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:36:59.443019 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:36:59.443019 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:36:59.443019 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:36:59.443019 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 21 22:36:59.443019 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:36:59.443019 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 21 22:36:59.443019 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:36:59.443019 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 577/1165] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Jul 21 22:36:59 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:36:59.447795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9016ce8e-2284-4a09-81a3-bf9c1abffa3f None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:36:59.536356 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:36:59.544659 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9016ce8e-2284-4a09-81a3-bf9c1abffa3f None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:36:59.577408 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cb7377-3aa5-4505-9304-750418ffb371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:36:59.578059 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cb7377-3aa5-4505-9304-750418ffb371 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:36:59.578341 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cb7377-3aa5-4505-9304-750418ffb371 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:36:59.578615 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cb7377-3aa5-4505-9304-750418ffb371 None admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:36:59.579435 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cb7377-3aa5-4505-9304-750418ffb371 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:36:59.580250 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cb7377-3aa5-4505-9304-750418ffb371 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:36:59.581602 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cb7377-3aa5-4505-9304-750418ffb371 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:36:59.588530 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 589/1166] 10.222.0.52 () {56 vars in 1115 bytes} [Thu Jul 21 22:36:59 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:37:01.588363 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70af2b7-6e40-4082-982a-1bd4c7628131 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:01.588856 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70af2b7-6e40-4082-982a-1bd4c7628131 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:01.588950 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70af2b7-6e40-4082-982a-1bd4c7628131 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:01.590290 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 578/1167] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:37:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:37:01.595551 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461276fa-b863-4529-a068-1a365d13703e None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:01.595920 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461276fa-b863-4529-a068-1a365d13703e None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:01.596096 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461276fa-b863-4529-a068-1a365d13703e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:01.612999 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-461276fa-b863-4529-a068-1a365d13703e None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:37:01.614163 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-461276fa-b863-4529-a068-1a365d13703e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:37:01.678025 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 590/1168] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:37:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:37:01.700565 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70af2b7-6e40-4082-982a-1bd4c7628131 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:01.801149 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70af2b7-6e40-4082-982a-1bd4c7628131 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:01.833923 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1019d2a-0be9-472c-95fa-36f3445dbcc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:01.834565 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1019d2a-0be9-472c-95fa-36f3445dbcc4 None admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:01.834877 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1019d2a-0be9-472c-95fa-36f3445dbcc4 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:01.835156 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1019d2a-0be9-472c-95fa-36f3445dbcc4 None admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:01.835899 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1019d2a-0be9-472c-95fa-36f3445dbcc4 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:37:01.837251 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1019d2a-0be9-472c-95fa-36f3445dbcc4 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:37:01.917953 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 579/1169] 10.222.0.52 () {60 vars in 1138 bytes} [Thu Jul 21 22:37:01 2022] POST /identity/v3/domains => generated 267 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:37:03.961553 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c082e3-68a9-4b99-a6b7-c29c07f22157 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:03.962113 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c082e3-68a9-4b99-a6b7-c29c07f22157 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:03.962113 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c082e3-68a9-4b99-a6b7-c29c07f22157 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:03.963517 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 591/1170] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:37:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:37:03.968699 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd6471-eaa8-4fc8-b582-bc5de79102e3 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:03.969102 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd6471-eaa8-4fc8-b582-bc5de79102e3 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:03.969249 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd6471-eaa8-4fc8-b582-bc5de79102e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:03.991314 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-2dcd6471-eaa8-4fc8-b582-bc5de79102e3 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:37:03.992497 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-2dcd6471-eaa8-4fc8-b582-bc5de79102e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:37:04.038044 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:37:04.056719 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 580/1171] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:37:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:37:04.081337 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c082e3-68a9-4b99-a6b7-c29c07f22157 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:04.165506 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:37:04.180607 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c082e3-68a9-4b99-a6b7-c29c07f22157 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:04.214669 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cc1410-849d-4e19-b903-3be0f9d3e967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:04.215489 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cc1410-849d-4e19-b903-3be0f9d3e967 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:04.215773 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cc1410-849d-4e19-b903-3be0f9d3e967 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:04.216050 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cc1410-849d-4e19-b903-3be0f9d3e967 None admin] PATH_INFO: `/v3/domains/46b130d623374beea823565e0d28c606` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:04.226499 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cc1410-849d-4e19-b903-3be0f9d3e967 None admin] RBAC: Authorizing `identity:get_domain(domain_id=46b130d623374beea823565e0d28c606)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:37:04.227974 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cc1410-849d-4e19-b903-3be0f9d3e967 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:37:04.229587 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 592/1172] 10.222.0.52 () {56 vars in 1183 bytes} [Thu Jul 21 22:37:04 2022] GET /identity/v3/domains/46b130d623374beea823565e0d28c606 => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:37:04.235463 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcd6471-eaa8-4fc8-b582-bc5de79102e3 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:04.332480 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcd6471-eaa8-4fc8-b582-bc5de79102e3 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:04.365124 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c954761-8f82-4ccb-9ce2-be666b933d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:04.365761 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c954761-8f82-4ccb-9ce2-be666b933d69 None admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:04.366040 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c954761-8f82-4ccb-9ce2-be666b933d69 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:04.366311 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c954761-8f82-4ccb-9ce2-be666b933d69 None admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:04.367038 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c954761-8f82-4ccb-9ce2-be666b933d69 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:37:04.368438 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c954761-8f82-4ccb-9ce2-be666b933d69 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:37:04.454765 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 581/1173] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jul 21 22:37:04 2022] POST /identity/v3/projects => generated 361 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:37:06.522440 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb84504-10ef-44b7-9fc5-570ea7c6703e None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:06.522948 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb84504-10ef-44b7-9fc5-570ea7c6703e None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:06.523054 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb84504-10ef-44b7-9fc5-570ea7c6703e None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:06.524363 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 593/1174] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:37:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:37:06.529504 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb442bf4-c791-489f-996a-e977b17f83f8 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:06.529908 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb442bf4-c791-489f-996a-e977b17f83f8 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:06.530054 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb442bf4-c791-489f-996a-e977b17f83f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:06.546224 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-fb442bf4-c791-489f-996a-e977b17f83f8 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:37:06.547420 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-fb442bf4-c791-489f-996a-e977b17f83f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:37:06.573147 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:37:06.615090 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 582/1175] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:37:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:37:06.639902 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb84504-10ef-44b7-9fc5-570ea7c6703e None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:06.740880 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb84504-10ef-44b7-9fc5-570ea7c6703e None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:06.773557 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b7369a-55cd-4657-b074-1a87901d76fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:06.774350 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7369a-55cd-4657-b074-1a87901d76fc None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:06.774633 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7369a-55cd-4657-b074-1a87901d76fc None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:06.774941 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7369a-55cd-4657-b074-1a87901d76fc None admin] PATH_INFO: `/v3/domains/46b130d623374beea823565e0d28c606` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:06.776116 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b7369a-55cd-4657-b074-1a87901d76fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=46b130d623374beea823565e0d28c606)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:37:06.777537 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b7369a-55cd-4657-b074-1a87901d76fc None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:37:06.779165 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 594/1176] 10.222.0.52 () {56 vars in 1183 bytes} [Thu Jul 21 22:37:06 2022] GET /identity/v3/domains/46b130d623374beea823565e0d28c606 => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:37:06.784868 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb442bf4-c791-489f-996a-e977b17f83f8 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:06.880741 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb442bf4-c791-489f-996a-e977b17f83f8 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:06.912911 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb5dfbc-eb2b-4057-bc33-36eb8833fa97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:06.913562 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb5dfbc-eb2b-4057-bc33-36eb8833fa97 None admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:06.913843 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb5dfbc-eb2b-4057-bc33-36eb8833fa97 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:06.914118 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb5dfbc-eb2b-4057-bc33-36eb8833fa97 None admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:06.914834 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb5dfbc-eb2b-4057-bc33-36eb8833fa97 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:37:06.916244 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb5dfbc-eb2b-4057-bc33-36eb8833fa97 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:37:07.006408 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 583/1177] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 21 22:37:06 2022] POST /identity/v3/users => generated 311 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:37:09.057273 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225d0091-a5f3-4d8f-97ea-c08438539091 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:09.057837 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225d0091-a5f3-4d8f-97ea-c08438539091 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:09.057943 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225d0091-a5f3-4d8f-97ea-c08438539091 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:09.059557 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 595/1178] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:37:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:37:09.064713 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece969c4-e08a-4b47-928d-c1530eeb3cdd None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:09.065131 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece969c4-e08a-4b47-928d-c1530eeb3cdd None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:09.065269 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece969c4-e08a-4b47-928d-c1530eeb3cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:09.082114 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-ece969c4-e08a-4b47-928d-c1530eeb3cdd None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:37:09.083317 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-ece969c4-e08a-4b47-928d-c1530eeb3cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:37:09.097456 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:37:09.147625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 584/1179] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:37:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:37:09.170898 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225d0091-a5f3-4d8f-97ea-c08438539091 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:09.186729 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:37:09.273082 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225d0091-a5f3-4d8f-97ea-c08438539091 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:09.305164 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec1d3c-bdcf-4f8a-aaf9-4618f387b302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:09.305943 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec1d3c-bdcf-4f8a-aaf9-4618f387b302 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:09.306227 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec1d3c-bdcf-4f8a-aaf9-4618f387b302 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:09.306503 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec1d3c-bdcf-4f8a-aaf9-4618f387b302 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:09.317436 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ec1d3c-bdcf-4f8a-aaf9-4618f387b302 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:37:09.325054 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-77ec1d3c-bdcf-4f8a-aaf9-4618f387b302 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:37:09.325054 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:37:09.325054 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:37:09.325054 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:37:09.325054 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:37:09.325054 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:37:09.325054 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:37:09.325054 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:37:09.325054 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:37:09.325054 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:37:09.325054 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:37:09.325054 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:37:09.325054 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:37:09.325054 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:37:09.325054 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:37:09.325054 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:37:09.326209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:37:09.326209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:37:09.326209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:37:09.326209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:37:09.326209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:37:09.326209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:37:09.326209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:37:09.326209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:37:09.326209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:37:09.326209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:37:09.326209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:37:09.326209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:37:09.326209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:37:09.326209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:37:09.326209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:37:09.326209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:37:09.327365 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:37:09.327365 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:37:09.327365 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:37:09.327365 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:37:09.327365 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:37:09.327365 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:37:09.327365 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:37:09.327365 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:37:09.327365 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:37:09.327365 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:37:09.327365 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:37:09.327365 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:37:09.327365 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 596/1180] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 22:37:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:37:09.332745 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece969c4-e08a-4b47-928d-c1530eeb3cdd None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:09.432553 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece969c4-e08a-4b47-928d-c1530eeb3cdd None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:09.465528 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7e7388-d9ad-4ce3-99c0-2aaae6b0778f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:09.466185 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e7388-d9ad-4ce3-99c0-2aaae6b0778f None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:09.466468 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e7388-d9ad-4ce3-99c0-2aaae6b0778f None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:09.466743 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e7388-d9ad-4ce3-99c0-2aaae6b0778f None admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:09.467510 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7e7388-d9ad-4ce3-99c0-2aaae6b0778f None admin] RBAC: Adding query filter params (name=admin) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:37:09.467816 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7e7388-d9ad-4ce3-99c0-2aaae6b0778f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:37:09.469165 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7e7388-d9ad-4ce3-99c0-2aaae6b0778f None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:37:09.477273 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 585/1181] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:37:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:37:09.482990 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec1d3c-bdcf-4f8a-aaf9-4618f387b302 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:09.555848 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:37:09.580310 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec1d3c-bdcf-4f8a-aaf9-4618f387b302 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:09.614841 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ee9ffc-427a-4b83-944a-e8d69bad6ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:09.615704 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee9ffc-427a-4b83-944a-e8d69bad6ccf None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:09.615987 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee9ffc-427a-4b83-944a-e8d69bad6ccf None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:09.616270 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee9ffc-427a-4b83-944a-e8d69bad6ccf None admin] PATH_INFO: `/v3/users/fa514b4b57984d9dadc6dd178b1976ea` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:09.629714 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ee9ffc-427a-4b83-944a-e8d69bad6ccf None admin] RBAC: Authorizing `identity:get_user(user_id=fa514b4b57984d9dadc6dd178b1976ea)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:37:09.631165 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ee9ffc-427a-4b83-944a-e8d69bad6ccf None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:37:09.632833 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 597/1182] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 22:37:09 2022] GET /identity/v3/users/fa514b4b57984d9dadc6dd178b1976ea => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:37:09.638593 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7e7388-d9ad-4ce3-99c0-2aaae6b0778f None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:09.654596 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:37:09.738573 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7e7388-d9ad-4ce3-99c0-2aaae6b0778f None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:09.772380 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ee5ec6-e8e4-4432-a92d-2840a7fdced8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:09.773017 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee5ec6-e8e4-4432-a92d-2840a7fdced8 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:09.773300 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee5ec6-e8e4-4432-a92d-2840a7fdced8 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:09.773578 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee5ec6-e8e4-4432-a92d-2840a7fdced8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:09.839101 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ee5ec6-e8e4-4432-a92d-2840a7fdced8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:37:09.840540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ee5ec6-e8e4-4432-a92d-2840a7fdced8 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:37:09.853785 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:37:09.936093 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 586/1183] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:37:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 22:37:09.942529 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ee9ffc-427a-4b83-944a-e8d69bad6ccf None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:10.038157 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ee9ffc-427a-4b83-944a-e8d69bad6ccf None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:10.068470 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:37:10.073165 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d60ba5-2328-4cbb-aecf-293acc2f8388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:10.074012 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d60ba5-2328-4cbb-aecf-293acc2f8388 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:10.074290 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d60ba5-2328-4cbb-aecf-293acc2f8388 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:10.074561 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d60ba5-2328-4cbb-aecf-293acc2f8388 None admin] PATH_INFO: `/v3/projects/5610e55bc40c4e128ec46a259d210895` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:10.075743 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d60ba5-2328-4cbb-aecf-293acc2f8388 None admin] RBAC: Authorizing `identity:get_project(project_id=5610e55bc40c4e128ec46a259d210895)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:37:10.077165 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d60ba5-2328-4cbb-aecf-293acc2f8388 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:37:10.078838 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 598/1184] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:37:09 2022] GET /identity/v3/projects/5610e55bc40c4e128ec46a259d210895 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:37:10.084894 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ee5ec6-e8e4-4432-a92d-2840a7fdced8 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:10.182583 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ee5ec6-e8e4-4432-a92d-2840a7fdced8 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:10.194803 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:37:10.215043 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820829df-65b5-4feb-9327-cbc3d5d28a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:10.215687 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820829df-65b5-4feb-9327-cbc3d5d28a0f None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:10.215971 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820829df-65b5-4feb-9327-cbc3d5d28a0f None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:10.216245 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820829df-65b5-4feb-9327-cbc3d5d28a0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:10.217073 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820829df-65b5-4feb-9327-cbc3d5d28a0f None admin] RBAC: Adding query filter params (role.id=d1ddee590a964ff1bbb4ee8716173021, scope.project.id=5610e55bc40c4e128ec46a259d210895, user.id=fa514b4b57984d9dadc6dd178b1976ea) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:37:10.217383 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820829df-65b5-4feb-9327-cbc3d5d28a0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:37:10.218814 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820829df-65b5-4feb-9327-cbc3d5d28a0f None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:37:10.224948 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 587/1185] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 22:37:10 2022] GET /identity/v3/role_assignments?user.id=fa514b4b57984d9dadc6dd178b1976ea&scope.project.id=5610e55bc40c4e128ec46a259d210895&role.id=d1ddee590a964ff1bbb4ee8716173021 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:37:12.243418 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af88ee-43e4-4681-916a-9f23791540c2 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:12.243975 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af88ee-43e4-4681-916a-9f23791540c2 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:12.243975 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af88ee-43e4-4681-916a-9f23791540c2 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:12.245329 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 599/1186] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:37:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:37:12.251220 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b57acab-5f71-4a0b-a09d-ab97d32c756a None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:12.251626 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b57acab-5f71-4a0b-a09d-ab97d32c756a None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:12.251777 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b57acab-5f71-4a0b-a09d-ab97d32c756a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:12.268063 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-0b57acab-5f71-4a0b-a09d-ab97d32c756a None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:37:12.269240 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-0b57acab-5f71-4a0b-a09d-ab97d32c756a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:37:12.335190 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 588/1187] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:37:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:37:12.358320 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41af88ee-43e4-4681-916a-9f23791540c2 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:12.457998 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41af88ee-43e4-4681-916a-9f23791540c2 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:12.491685 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c50384-e23c-44e0-8dc5-6f9890ad537a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:12.492461 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c50384-e23c-44e0-8dc5-6f9890ad537a None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:12.492745 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c50384-e23c-44e0-8dc5-6f9890ad537a None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:12.493017 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c50384-e23c-44e0-8dc5-6f9890ad537a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:12.503170 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c50384-e23c-44e0-8dc5-6f9890ad537a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:37:12.510654 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-c9c50384-e23c-44e0-8dc5-6f9890ad537a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:37:12.510654 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:37:12.510654 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:37:12.510654 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:37:12.510654 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:37:12.510654 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:37:12.510654 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:37:12.510654 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:37:12.510654 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:37:12.510654 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:37:12.510654 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:37:12.510654 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:37:12.510654 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:37:12.510654 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:37:12.510654 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:37:12.510654 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:37:12.511770 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:37:12.511770 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:37:12.511770 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:37:12.511770 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:37:12.511770 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:37:12.511770 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:37:12.511770 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:37:12.511770 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:37:12.511770 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:37:12.511770 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:37:12.511770 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:37:12.511770 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:37:12.511770 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:37:12.511770 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:37:12.511770 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:37:12.511770 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:37:12.513033 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:37:12.513033 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:37:12.513033 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:37:12.513033 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:37:12.513033 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:37:12.513033 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:37:12.513033 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:37:12.513033 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:37:12.513033 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:37:12.513033 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:37:12.513033 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:37:12.513033 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:37:12.513033 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 600/1188] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 22:37:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:37:12.517885 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b57acab-5f71-4a0b-a09d-ab97d32c756a None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:12.615706 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b57acab-5f71-4a0b-a09d-ab97d32c756a None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:12.622761 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:37:12.649507 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66265c1b-e3ea-4b1e-8292-ae31ec84f200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:12.650167 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66265c1b-e3ea-4b1e-8292-ae31ec84f200 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:12.650452 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66265c1b-e3ea-4b1e-8292-ae31ec84f200 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:12.650727 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66265c1b-e3ea-4b1e-8292-ae31ec84f200 None admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:12.651499 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66265c1b-e3ea-4b1e-8292-ae31ec84f200 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:37:12.651806 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66265c1b-e3ea-4b1e-8292-ae31ec84f200 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:37:12.653164 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66265c1b-e3ea-4b1e-8292-ae31ec84f200 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:37:12.662057 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 589/1189] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:37:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:37:12.668040 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c50384-e23c-44e0-8dc5-6f9890ad537a None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:12.727206 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:37:12.767595 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c50384-e23c-44e0-8dc5-6f9890ad537a None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:12.799995 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01972ee-d7d5-4f30-920e-3234895bf9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:12.800849 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01972ee-d7d5-4f30-920e-3234895bf9da None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:12.801134 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01972ee-d7d5-4f30-920e-3234895bf9da None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:12.801411 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01972ee-d7d5-4f30-920e-3234895bf9da None admin] PATH_INFO: `/v3/users/fa514b4b57984d9dadc6dd178b1976ea` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:12.802587 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01972ee-d7d5-4f30-920e-3234895bf9da None admin] RBAC: Authorizing `identity:get_user(user_id=fa514b4b57984d9dadc6dd178b1976ea)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:37:12.804035 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01972ee-d7d5-4f30-920e-3234895bf9da None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:37:12.805640 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 601/1190] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 22:37:12 2022] GET /identity/v3/users/fa514b4b57984d9dadc6dd178b1976ea => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:37:12.811409 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66265c1b-e3ea-4b1e-8292-ae31ec84f200 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:12.908511 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66265c1b-e3ea-4b1e-8292-ae31ec84f200 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:12.940119 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558824a5-3678-4884-aaac-a4d2b5355708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:12.940952 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558824a5-3678-4884-aaac-a4d2b5355708 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:12.941234 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558824a5-3678-4884-aaac-a4d2b5355708 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:12.941506 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558824a5-3678-4884-aaac-a4d2b5355708 None admin] PATH_INFO: `/v3/projects/5610e55bc40c4e128ec46a259d210895` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:12.942664 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558824a5-3678-4884-aaac-a4d2b5355708 None admin] RBAC: Authorizing `identity:get_project(project_id=5610e55bc40c4e128ec46a259d210895)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:37:12.944118 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558824a5-3678-4884-aaac-a4d2b5355708 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:37:12.945769 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 590/1191] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:37:12 2022] GET /identity/v3/projects/5610e55bc40c4e128ec46a259d210895 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:37:12.951574 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01972ee-d7d5-4f30-920e-3234895bf9da None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:13.049412 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01972ee-d7d5-4f30-920e-3234895bf9da None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:13.074486 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:37:13.082618 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb607b1-6b61-4ef5-b0aa-066d86811018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:13.088649 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb607b1-6b61-4ef5-b0aa-066d86811018 None admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:13.088649 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb607b1-6b61-4ef5-b0aa-066d86811018 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:13.088649 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb607b1-6b61-4ef5-b0aa-066d86811018 None admin] PATH_INFO: `/v3/projects/5610e55bc40c4e128ec46a259d210895/users/fa514b4b57984d9dadc6dd178b1976ea/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:13.088649 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb607b1-6b61-4ef5-b0aa-066d86811018 None admin] RBAC: Authorizing `identity:create_grant(project_id=5610e55bc40c4e128ec46a259d210895, user_id=fa514b4b57984d9dadc6dd178b1976ea, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:37:13.088649 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb607b1-6b61-4ef5-b0aa-066d86811018 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:37:13.173113 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 602/1192] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 21 22:37:12 2022] PUT /identity/v3/projects/5610e55bc40c4e128ec46a259d210895/users/fa514b4b57984d9dadc6dd178b1976ea/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:37:15.161207 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3416f8-d883-4958-9bab-f976730fed9d None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:15.161754 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3416f8-d883-4958-9bab-f976730fed9d None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:15.161754 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3416f8-d883-4958-9bab-f976730fed9d None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:15.163160 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 591/1193] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:37:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:37:15.168468 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f8f1d-e670-4511-850e-b1553b16ef75 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:15.168855 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f8f1d-e670-4511-850e-b1553b16ef75 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:15.169030 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f8f1d-e670-4511-850e-b1553b16ef75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:15.185933 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-e75f8f1d-e670-4511-850e-b1553b16ef75 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:37:15.187156 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-e75f8f1d-e670-4511-850e-b1553b16ef75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:37:15.252169 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 603/1194] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:37:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:37:15.274899 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3416f8-d883-4958-9bab-f976730fed9d None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:15.313292 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:37:15.377896 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3416f8-d883-4958-9bab-f976730fed9d None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:15.411353 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c31bee-4d6c-4e77-94e7-a0bbc0c3fff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:15.412071 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c31bee-4d6c-4e77-94e7-a0bbc0c3fff5 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:15.412355 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c31bee-4d6c-4e77-94e7-a0bbc0c3fff5 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:15.412629 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c31bee-4d6c-4e77-94e7-a0bbc0c3fff5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:15.422505 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c31bee-4d6c-4e77-94e7-a0bbc0c3fff5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:37:15.430214 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-c6c31bee-4d6c-4e77-94e7-a0bbc0c3fff5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:37:15.430214 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:37:15.430214 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:37:15.430214 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:37:15.430214 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:37:15.430214 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:37:15.430214 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:37:15.430214 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:37:15.430214 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:37:15.430214 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:37:15.430214 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:37:15.430214 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:37:15.430214 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:37:15.430214 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:37:15.430214 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:37:15.430214 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:37:15.431275 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:37:15.431275 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:37:15.431275 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:37:15.431275 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:37:15.431275 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:37:15.431275 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:37:15.431275 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:37:15.431275 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:37:15.431275 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:37:15.431275 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:37:15.431275 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:37:15.431275 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:37:15.431275 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:37:15.431275 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:37:15.431275 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:37:15.431275 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:37:15.432187 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:37:15.432187 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:37:15.432187 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:37:15.432187 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:37:15.432187 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:37:15.432187 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:37:15.432187 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:37:15.432187 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:37:15.432187 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:37:15.432187 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:37:15.432187 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:37:15.432187 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:37:15.432187 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 592/1195] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 22:37:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:37:15.437407 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f8f1d-e670-4511-850e-b1553b16ef75 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:15.534383 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f8f1d-e670-4511-850e-b1553b16ef75 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:15.550999 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:37:15.567113 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9241a9-b5f9-4bc6-97fc-1b7a7943abac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:15.567775 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9241a9-b5f9-4bc6-97fc-1b7a7943abac None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:15.568053 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9241a9-b5f9-4bc6-97fc-1b7a7943abac None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:15.568325 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9241a9-b5f9-4bc6-97fc-1b7a7943abac None admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:15.569061 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9241a9-b5f9-4bc6-97fc-1b7a7943abac None admin] RBAC: Adding query filter params (name=admin) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:37:15.569364 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9241a9-b5f9-4bc6-97fc-1b7a7943abac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:37:15.570710 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9241a9-b5f9-4bc6-97fc-1b7a7943abac None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:37:15.579307 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 604/1196] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:37:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:37:15.585018 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c31bee-4d6c-4e77-94e7-a0bbc0c3fff5 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:15.678099 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:37:15.683763 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c31bee-4d6c-4e77-94e7-a0bbc0c3fff5 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:15.717511 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf3548a-0c1b-4aa2-b874-e99c03cded5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:15.718369 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3548a-0c1b-4aa2-b874-e99c03cded5f None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:15.718655 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3548a-0c1b-4aa2-b874-e99c03cded5f None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:15.718971 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3548a-0c1b-4aa2-b874-e99c03cded5f None admin] PATH_INFO: `/v3/users/fa514b4b57984d9dadc6dd178b1976ea` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:15.720155 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf3548a-0c1b-4aa2-b874-e99c03cded5f None admin] RBAC: Authorizing `identity:get_user(user_id=fa514b4b57984d9dadc6dd178b1976ea)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:37:15.721590 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf3548a-0c1b-4aa2-b874-e99c03cded5f None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:37:15.723247 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 593/1197] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Jul 21 22:37:15 2022] GET /identity/v3/users/fa514b4b57984d9dadc6dd178b1976ea => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:37:15.729302 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9241a9-b5f9-4bc6-97fc-1b7a7943abac None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:15.825004 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9241a9-b5f9-4bc6-97fc-1b7a7943abac None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:15.856776 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91e202-2094-4274-be02-7425d050b34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:15.857412 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91e202-2094-4274-be02-7425d050b34a None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:15.857690 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91e202-2094-4274-be02-7425d050b34a None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:15.857962 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91e202-2094-4274-be02-7425d050b34a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:15.905075 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:37:15.924134 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc91e202-2094-4274-be02-7425d050b34a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:37:15.925568 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc91e202-2094-4274-be02-7425d050b34a None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:37:16.020094 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 605/1198] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:37:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 22:37:16.025828 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf3548a-0c1b-4aa2-b874-e99c03cded5f None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:16.123354 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf3548a-0c1b-4aa2-b874-e99c03cded5f None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:16.155845 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccc7c29-7c0f-4033-b8de-998a292beaf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:16.157197 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccc7c29-7c0f-4033-b8de-998a292beaf3 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:16.157476 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccc7c29-7c0f-4033-b8de-998a292beaf3 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:16.157750 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccc7c29-7c0f-4033-b8de-998a292beaf3 None admin] PATH_INFO: `/v3/projects/5610e55bc40c4e128ec46a259d210895` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:16.158941 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccc7c29-7c0f-4033-b8de-998a292beaf3 None admin] RBAC: Authorizing `identity:get_project(project_id=5610e55bc40c4e128ec46a259d210895)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:37:16.160349 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccc7c29-7c0f-4033-b8de-998a292beaf3 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:37:16.162033 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 594/1199] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Jul 21 22:37:16 2022] GET /identity/v3/projects/5610e55bc40c4e128ec46a259d210895 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:37:16.168099 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91e202-2094-4274-be02-7425d050b34a None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:37:16.246638 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:37:16.264930 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91e202-2094-4274-be02-7425d050b34a None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:37:16.299233 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bf435c-adf2-4621-ab57-1c97db85c4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:37:16.299819 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bf435c-adf2-4621-ab57-1c97db85c4ee None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:37:16.300102 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bf435c-adf2-4621-ab57-1c97db85c4ee None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:37:16.300377 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bf435c-adf2-4621-ab57-1c97db85c4ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:37:16.301200 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bf435c-adf2-4621-ab57-1c97db85c4ee None admin] RBAC: Adding query filter params (role.id=d1ddee590a964ff1bbb4ee8716173021, scope.project.id=5610e55bc40c4e128ec46a259d210895, user.id=fa514b4b57984d9dadc6dd178b1976ea) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:37:16.301511 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bf435c-adf2-4621-ab57-1c97db85c4ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:37:16.302956 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bf435c-adf2-4621-ab57-1c97db85c4ee None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:37:16.308665 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 606/1200] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 22:37:16 2022] GET /identity/v3/role_assignments?user.id=fa514b4b57984d9dadc6dd178b1976ea&scope.project.id=5610e55bc40c4e128ec46a259d210895&role.id=d1ddee590a964ff1bbb4ee8716173021 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:37:26.151723 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=84 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:37:26.306072 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=28 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:27.789374 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce9b30-10f6-4741-aca3-c3ecc3140667 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:27.789916 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce9b30-10f6-4741-aca3-c3ecc3140667 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:27.789916 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce9b30-10f6-4741-aca3-c3ecc3140667 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:27.791278 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 595/1201] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:38:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:38:27.796547 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e9c40-822c-4728-b453-35579ca3c9df None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:27.796969 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e9c40-822c-4728-b453-35579ca3c9df None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:27.797104 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e9c40-822c-4728-b453-35579ca3c9df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:27.814643 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-d44e9c40-822c-4728-b453-35579ca3c9df None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:38:27.815862 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-d44e9c40-822c-4728-b453-35579ca3c9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:27.882250 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 607/1202] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:38:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:38:27.908470 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ce9b30-10f6-4741-aca3-c3ecc3140667 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:28.008486 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ce9b30-10f6-4741-aca3-c3ecc3140667 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:28.043342 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9653744-b798-45e4-b5b5-12cbb6b5704c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:28.044060 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9653744-b798-45e4-b5b5-12cbb6b5704c None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:28.044358 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9653744-b798-45e4-b5b5-12cbb6b5704c None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:28.044641 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9653744-b798-45e4-b5b5-12cbb6b5704c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:28.055631 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9653744-b798-45e4-b5b5-12cbb6b5704c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:28.057122 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9653744-b798-45e4-b5b5-12cbb6b5704c None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:28.074466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-d9653744-b798-45e4-b5b5-12cbb6b5704c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:38:28.074466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:38:28.074466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:38:28.074466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:38:28.074466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:38:28.074466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:38:28.074466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:38:28.074466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:38:28.074466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:38:28.074466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:38:28.074466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:38:28.074466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:38:28.074466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:38:28.074466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:38:28.074466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:38:28.076377 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:38:28.076377 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:38:28.076377 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:38:28.076377 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:38:28.076377 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:38:28.076377 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:38:28.076377 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:38:28.076377 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:38:28.076377 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:38:28.076377 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:38:28.076377 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:38:28.076377 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:38:28.076377 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:38:28.076377 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:38:28.076377 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:38:28.076377 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:38:28.076377 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:38:28.078195 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:38:28.078195 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:38:28.078195 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:38:28.078195 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:38:28.078195 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 596/1203] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:38:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:38:28.081989 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44e9c40-822c-4728-b453-35579ca3c9df None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:28.183728 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44e9c40-822c-4728-b453-35579ca3c9df None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:28.211691 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:28.217141 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80287755-981e-451e-8ddb-809bf83aea4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:28.217802 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80287755-981e-451e-8ddb-809bf83aea4e None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:28.218089 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80287755-981e-451e-8ddb-809bf83aea4e None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:28.218369 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80287755-981e-451e-8ddb-809bf83aea4e None admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:28.219633 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80287755-981e-451e-8ddb-809bf83aea4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:38:28.219940 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80287755-981e-451e-8ddb-809bf83aea4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:28.221306 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80287755-981e-451e-8ddb-809bf83aea4e None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:28.232958 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 608/1204] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:38:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:38:28.239594 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9653744-b798-45e4-b5b5-12cbb6b5704c None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:28.292569 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:28.338127 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9653744-b798-45e4-b5b5-12cbb6b5704c None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:28.371986 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0635420b-e115-47c5-967a-0cb8d637215a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:28.372647 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635420b-e115-47c5-967a-0cb8d637215a None admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:28.372929 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635420b-e115-47c5-967a-0cb8d637215a None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:28.373201 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635420b-e115-47c5-967a-0cb8d637215a None admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:28.373893 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0635420b-e115-47c5-967a-0cb8d637215a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:28.375377 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0635420b-e115-47c5-967a-0cb8d637215a None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:28.486628 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 597/1205] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Jul 21 22:38:28 2022] POST /identity/v3/users => generated 251 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:38:30.556894 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b069f04-f74c-4d4b-b5d2-98a35fbabe85 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:30.557424 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b069f04-f74c-4d4b-b5d2-98a35fbabe85 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:30.557529 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b069f04-f74c-4d4b-b5d2-98a35fbabe85 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:30.558834 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 609/1206] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:38:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:38:30.564749 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d67f8e7-784d-4efb-9385-84b4b231b2c8 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:30.565204 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d67f8e7-784d-4efb-9385-84b4b231b2c8 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:30.565351 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d67f8e7-784d-4efb-9385-84b4b231b2c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:30.581740 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-0d67f8e7-784d-4efb-9385-84b4b231b2c8 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:38:30.582962 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-0d67f8e7-784d-4efb-9385-84b4b231b2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:30.650481 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 598/1207] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:38:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:38:30.674563 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b069f04-f74c-4d4b-b5d2-98a35fbabe85 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:30.782559 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b069f04-f74c-4d4b-b5d2-98a35fbabe85 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:30.815533 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ca0d66-c0bf-43fb-8772-4dfdad36e085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:30.819860 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca0d66-c0bf-43fb-8772-4dfdad36e085 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:30.819860 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca0d66-c0bf-43fb-8772-4dfdad36e085 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:30.819860 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca0d66-c0bf-43fb-8772-4dfdad36e085 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:30.827618 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ca0d66-c0bf-43fb-8772-4dfdad36e085 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:38:30.835901 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-39ca0d66-c0bf-43fb-8772-4dfdad36e085 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:38:30.835901 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:38:30.835901 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:38:30.835901 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:38:30.835901 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:38:30.835901 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:38:30.835901 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:38:30.835901 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:38:30.835901 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:38:30.835901 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:38:30.835901 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:38:30.835901 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:38:30.835901 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:38:30.835901 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:38:30.835901 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:38:30.835901 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:38:30.837019 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:38:30.837019 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:38:30.837019 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:38:30.837019 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:38:30.837019 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:38:30.837019 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:38:30.837019 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:38:30.837019 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:38:30.837019 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:38:30.837019 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:38:30.837019 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:38:30.837019 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:38:30.837019 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:38:30.837019 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:38:30.837019 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:38:30.837019 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:38:30.837954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:38:30.837954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:38:30.837954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:38:30.837954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:38:30.837954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:38:30.837954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:38:30.837954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:38:30.837954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:38:30.837954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:38:30.837954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:38:30.837954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:38:30.837954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:38:30.837954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 610/1208] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 21 22:38:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:38:30.843202 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d67f8e7-784d-4efb-9385-84b4b231b2c8 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:30.884895 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:30.943303 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d67f8e7-784d-4efb-9385-84b4b231b2c8 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:30.976688 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850289a-bfdb-4c57-a909-43c839c282ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:30.977357 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850289a-bfdb-4c57-a909-43c839c282ef None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:30.977643 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850289a-bfdb-4c57-a909-43c839c282ef None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:30.977915 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850289a-bfdb-4c57-a909-43c839c282ef None admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:30.978657 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4850289a-bfdb-4c57-a909-43c839c282ef None admin] RBAC: Adding query filter params (name=service) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:38:30.978991 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4850289a-bfdb-4c57-a909-43c839c282ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:30.980345 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4850289a-bfdb-4c57-a909-43c839c282ef None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:30.988175 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 599/1209] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 21 22:38:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:38:30.993996 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ca0d66-c0bf-43fb-8772-4dfdad36e085 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:31.046133 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:31.096442 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ca0d66-c0bf-43fb-8772-4dfdad36e085 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:31.129993 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94da2db-dfb8-4673-a888-7c9515c1f4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:31.130800 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94da2db-dfb8-4673-a888-7c9515c1f4bc None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:31.131088 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94da2db-dfb8-4673-a888-7c9515c1f4bc None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:31.131364 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94da2db-dfb8-4673-a888-7c9515c1f4bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:31.141814 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94da2db-dfb8-4673-a888-7c9515c1f4bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:31.143295 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94da2db-dfb8-4673-a888-7c9515c1f4bc None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:31.161089 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-e94da2db-dfb8-4673-a888-7c9515c1f4bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:38:31.161089 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:38:31.161089 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:38:31.161089 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:38:31.161089 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:38:31.161089 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:38:31.161089 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:38:31.161089 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:38:31.161089 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:38:31.161089 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:38:31.161089 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:38:31.161089 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:38:31.161089 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:38:31.161089 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:38:31.161089 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:38:31.164582 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:38:31.164582 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:38:31.164582 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:38:31.164582 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:38:31.164582 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:38:31.164582 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:38:31.164582 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:38:31.164582 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:38:31.164582 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:38:31.164582 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:38:31.164582 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:38:31.164582 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:38:31.164582 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:38:31.164582 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:38:31.164582 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:38:31.164582 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:38:31.164582 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:38:31.165699 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:38:31.165699 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:38:31.165699 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:38:31.165699 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:38:31.165699 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 611/1210] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:38:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:38:31.168382 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850289a-bfdb-4c57-a909-43c839c282ef None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:31.265173 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:31.268757 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850289a-bfdb-4c57-a909-43c839c282ef None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:31.304928 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02edca57-a785-4f0c-a487-abea183d5983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:31.305564 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02edca57-a785-4f0c-a487-abea183d5983 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:31.305838 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02edca57-a785-4f0c-a487-abea183d5983 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:31.306108 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02edca57-a785-4f0c-a487-abea183d5983 None admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:31.306892 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02edca57-a785-4f0c-a487-abea183d5983 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:38:31.307215 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02edca57-a785-4f0c-a487-abea183d5983 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:31.310151 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02edca57-a785-4f0c-a487-abea183d5983 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:31.320413 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 600/1211] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:38:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:38:31.326270 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94da2db-dfb8-4673-a888-7c9515c1f4bc None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:31.425606 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94da2db-dfb8-4673-a888-7c9515c1f4bc None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:31.427160 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:31.459307 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d34461a-b0f4-4cd9-894d-700f3574452e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:31.460099 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d34461a-b0f4-4cd9-894d-700f3574452e None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:31.460380 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d34461a-b0f4-4cd9-894d-700f3574452e None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:31.460657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d34461a-b0f4-4cd9-894d-700f3574452e None admin] PATH_INFO: `/v3/users/placement` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:31.473496 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d34461a-b0f4-4cd9-894d-700f3574452e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:31.475045 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d34461a-b0f4-4cd9-894d-700f3574452e None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:31.492190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-2d34461a-b0f4-4cd9-894d-700f3574452e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 22:38:31.492190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:38:31.492190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:38:31.492190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:38:31.492190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:38:31.492190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:38:31.492190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:38:31.492190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:38:31.492190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:38:31.492190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:38:31.492190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:38:31.492190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:38:31.492190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:38:31.492190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:38:31.492190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:38:31.492190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:38:31.493209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:38:31.493209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:38:31.493209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:38:31.493209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:38:31.493209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:38:31.493209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:38:31.493209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:38:31.493209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:38:31.493209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:38:31.493209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:38:31.493209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:38:31.493209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:38:31.493209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:38:31.493209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:38:31.493209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:38:31.493209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:38:31.493209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:38:31.493209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:38:31.494213 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:38:31.494213 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:38:31.494213 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:38:31.494213 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:38:31.494213 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:38:31.494213 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:38:31.494213 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:38:31.494213 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:38:31.494213 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 22:38:31.494213 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:38:31.494213 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 612/1212] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:38:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:38:31.499405 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02edca57-a785-4f0c-a487-abea183d5983 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:31.603216 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02edca57-a785-4f0c-a487-abea183d5983 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:31.637854 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aac3cb9-bf5a-4c70-a921-7a6b4dafa775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:31.638536 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac3cb9-bf5a-4c70-a921-7a6b4dafa775 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:31.638856 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac3cb9-bf5a-4c70-a921-7a6b4dafa775 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:31.639155 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac3cb9-bf5a-4c70-a921-7a6b4dafa775 None admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:31.640046 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aac3cb9-bf5a-4c70-a921-7a6b4dafa775 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:38:31.640354 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aac3cb9-bf5a-4c70-a921-7a6b4dafa775 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:31.641830 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aac3cb9-bf5a-4c70-a921-7a6b4dafa775 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:31.665198 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 601/1213] 10.222.0.52 () {56 vars in 1143 bytes} [Thu Jul 21 22:38:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:38:31.666334 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d34461a-b0f4-4cd9-894d-700f3574452e None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:31.769075 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d34461a-b0f4-4cd9-894d-700f3574452e None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:31.803287 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04c6a45-e493-4921-afaf-f072e242c40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:31.803928 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c6a45-e493-4921-afaf-f072e242c40c None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:31.804209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c6a45-e493-4921-afaf-f072e242c40c None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:31.804483 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c6a45-e493-4921-afaf-f072e242c40c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:31.833273 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:31.871625 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04c6a45-e493-4921-afaf-f072e242c40c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:31.872990 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04c6a45-e493-4921-afaf-f072e242c40c None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:31.976738 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 613/1214] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:38:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 22:38:31.982608 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aac3cb9-bf5a-4c70-a921-7a6b4dafa775 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:32.001886 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:32.082338 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aac3cb9-bf5a-4c70-a921-7a6b4dafa775 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:32.121067 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9497a3d5-9803-4c51-a164-372059cdd269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:32.121866 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9497a3d5-9803-4c51-a164-372059cdd269 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:32.122098 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9497a3d5-9803-4c51-a164-372059cdd269 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:32.122365 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9497a3d5-9803-4c51-a164-372059cdd269 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:32.133171 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9497a3d5-9803-4c51-a164-372059cdd269 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:32.134695 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9497a3d5-9803-4c51-a164-372059cdd269 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:32.151712 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-9497a3d5-9803-4c51-a164-372059cdd269 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:38:32.151712 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:38:32.151712 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:38:32.151712 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:38:32.151712 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:38:32.151712 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:38:32.151712 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:38:32.151712 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:38:32.151712 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:38:32.151712 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:38:32.151712 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:38:32.151712 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:38:32.151712 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:38:32.151712 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:38:32.151712 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:38:32.157320 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:38:32.157320 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:38:32.157320 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:38:32.157320 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:38:32.157320 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:38:32.157320 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:38:32.157320 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:38:32.157320 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:38:32.157320 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:38:32.157320 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:38:32.157320 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:38:32.157320 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:38:32.157320 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:38:32.157320 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:38:32.157320 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:38:32.157320 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:38:32.157320 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:38:32.158286 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:38:32.158286 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:38:32.158286 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:38:32.158286 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:38:32.158286 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 602/1215] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:38:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:38:32.158793 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04c6a45-e493-4921-afaf-f072e242c40c None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:32.219701 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:32.257824 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04c6a45-e493-4921-afaf-f072e242c40c None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:32.291466 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d46c32f-7e43-4b52-8edf-27b7d1d72d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:32.292122 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d46c32f-7e43-4b52-8edf-27b7d1d72d18 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:32.292403 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d46c32f-7e43-4b52-8edf-27b7d1d72d18 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:32.292676 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d46c32f-7e43-4b52-8edf-27b7d1d72d18 None admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:32.293389 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d46c32f-7e43-4b52-8edf-27b7d1d72d18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:38:32.293695 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d46c32f-7e43-4b52-8edf-27b7d1d72d18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:32.295073 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d46c32f-7e43-4b52-8edf-27b7d1d72d18 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:32.309239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 614/1216] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:38:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:38:32.315600 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9497a3d5-9803-4c51-a164-372059cdd269 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:32.383774 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:32.416450 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9497a3d5-9803-4c51-a164-372059cdd269 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:32.450179 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99818b4-81eb-428a-b02c-493aa736825d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:32.451020 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99818b4-81eb-428a-b02c-493aa736825d None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:32.451505 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99818b4-81eb-428a-b02c-493aa736825d None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:32.456712 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99818b4-81eb-428a-b02c-493aa736825d None admin] PATH_INFO: `/v3/projects/service` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:32.459105 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99818b4-81eb-428a-b02c-493aa736825d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:32.460527 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99818b4-81eb-428a-b02c-493aa736825d None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:32.475191 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-c99818b4-81eb-428a-b02c-493aa736825d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:38:32.475191 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:38:32.475191 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:38:32.475191 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:38:32.475191 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:38:32.475191 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:38:32.475191 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:38:32.475191 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:38:32.475191 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:38:32.475191 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:38:32.475191 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:38:32.475191 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:38:32.475191 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:38:32.475191 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:38:32.475191 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:38:32.476151 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:38:32.476151 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:38:32.476151 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:38:32.476151 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:38:32.476151 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:38:32.476151 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:38:32.476151 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:38:32.476151 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:38:32.476151 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:38:32.476151 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:38:32.476151 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:38:32.476151 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:38:32.476151 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:38:32.476151 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:38:32.476151 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:38:32.476151 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:38:32.476151 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:38:32.477040 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:38:32.477040 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:38:32.477040 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:38:32.477040 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:38:32.477040 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:38:32.477040 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:38:32.477040 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:38:32.477040 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:38:32.477040 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 603/1217] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:38:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:38:32.482343 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d46c32f-7e43-4b52-8edf-27b7d1d72d18 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:32.581362 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d46c32f-7e43-4b52-8edf-27b7d1d72d18 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:32.603771 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:32.615540 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381d7ce8-e005-4da4-8e78-c1b5ce9c6591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:32.616191 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d7ce8-e005-4da4-8e78-c1b5ce9c6591 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:32.616526 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d7ce8-e005-4da4-8e78-c1b5ce9c6591 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:32.616796 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d7ce8-e005-4da4-8e78-c1b5ce9c6591 None admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:32.617570 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381d7ce8-e005-4da4-8e78-c1b5ce9c6591 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:38:32.617874 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381d7ce8-e005-4da4-8e78-c1b5ce9c6591 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:32.619324 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381d7ce8-e005-4da4-8e78-c1b5ce9c6591 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:32.631103 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 615/1218] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 22:38:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:38:32.637417 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99818b4-81eb-428a-b02c-493aa736825d None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:32.736450 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99818b4-81eb-428a-b02c-493aa736825d None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:32.764897 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:32.771103 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6ac49e-56c9-406a-9df0-8567b165c6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:32.771751 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ac49e-56c9-406a-9df0-8567b165c6f8 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:32.772032 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ac49e-56c9-406a-9df0-8567b165c6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:32.772303 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ac49e-56c9-406a-9df0-8567b165c6f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:32.773130 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6ac49e-56c9-406a-9df0-8567b165c6f8 None admin] RBAC: Adding query filter params (role.id=715d866890d14923b11470d544b4f960, scope.project.id=aa11ab8229b642ce962193b74001d3bc, user.id=10c0bed169344a77bfad29624c5e5a8e) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:38:32.773439 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6ac49e-56c9-406a-9df0-8567b165c6f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:32.774864 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6ac49e-56c9-406a-9df0-8567b165c6f8 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:32.780201 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 604/1219] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 22:38:32 2022] GET /identity/v3/role_assignments?user.id=10c0bed169344a77bfad29624c5e5a8e&scope.project.id=aa11ab8229b642ce962193b74001d3bc&role.id=715d866890d14923b11470d544b4f960 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:38:34.774252 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab32a9-fec5-4e98-aff2-62443393b913 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:34.774750 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab32a9-fec5-4e98-aff2-62443393b913 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:34.774750 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab32a9-fec5-4e98-aff2-62443393b913 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:34.776111 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 616/1220] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:38:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:38:34.781362 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb80cc-9392-42f0-9783-93a2076064d4 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:34.782370 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb80cc-9392-42f0-9783-93a2076064d4 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:34.782471 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb80cc-9392-42f0-9783-93a2076064d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:34.799360 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-c1eb80cc-9392-42f0-9783-93a2076064d4 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:38:34.800538 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-c1eb80cc-9392-42f0-9783-93a2076064d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:34.867655 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 605/1221] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:38:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:38:34.890502 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ab32a9-fec5-4e98-aff2-62443393b913 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:34.990350 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ab32a9-fec5-4e98-aff2-62443393b913 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:35.026814 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df6e384-305c-4805-9560-2c18b1255707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:35.027548 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df6e384-305c-4805-9560-2c18b1255707 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:35.027829 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df6e384-305c-4805-9560-2c18b1255707 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:35.028103 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df6e384-305c-4805-9560-2c18b1255707 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:35.038450 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df6e384-305c-4805-9560-2c18b1255707 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:38:35.046247 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-0df6e384-305c-4805-9560-2c18b1255707 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:38:35.046247 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:38:35.046247 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:38:35.046247 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:38:35.046247 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:38:35.046247 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:38:35.046247 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:38:35.046247 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:38:35.046247 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:38:35.046247 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:38:35.046247 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:38:35.046247 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:38:35.046247 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:38:35.046247 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:38:35.046247 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:38:35.046247 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:38:35.051605 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:38:35.051605 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:38:35.051605 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:38:35.051605 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:38:35.051605 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:38:35.051605 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:38:35.051605 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:38:35.051605 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:38:35.051605 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:38:35.051605 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:38:35.051605 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:38:35.051605 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:38:35.051605 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:38:35.051605 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:38:35.051605 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:38:35.051605 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:38:35.052460 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:38:35.052460 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:38:35.052460 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:38:35.052460 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:38:35.052460 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:38:35.052460 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:38:35.052460 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:38:35.052460 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:38:35.052460 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:38:35.052460 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:38:35.052460 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:38:35.052460 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:38:35.052460 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 617/1222] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 21 22:38:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:38:35.053975 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb80cc-9392-42f0-9783-93a2076064d4 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:35.153976 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb80cc-9392-42f0-9783-93a2076064d4 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:35.177111 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:35.187343 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b12f8-871d-4a56-b63c-67791d21fdc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:35.188738 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b12f8-871d-4a56-b63c-67791d21fdc7 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:35.189033 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b12f8-871d-4a56-b63c-67791d21fdc7 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:35.189309 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b12f8-871d-4a56-b63c-67791d21fdc7 None admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:35.190043 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b12f8-871d-4a56-b63c-67791d21fdc7 None admin] RBAC: Adding query filter params (name=service) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:38:35.190343 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b12f8-871d-4a56-b63c-67791d21fdc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:35.191732 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b12f8-871d-4a56-b63c-67791d21fdc7 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:35.199576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 606/1223] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 21 22:38:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:38:35.205542 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df6e384-305c-4805-9560-2c18b1255707 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:35.249173 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:35.303413 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df6e384-305c-4805-9560-2c18b1255707 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:35.335350 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdf5d78-ad51-46d9-bc06-477d9c4bd132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:35.336125 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdf5d78-ad51-46d9-bc06-477d9c4bd132 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:35.336404 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdf5d78-ad51-46d9-bc06-477d9c4bd132 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:35.336676 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdf5d78-ad51-46d9-bc06-477d9c4bd132 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:35.349198 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdf5d78-ad51-46d9-bc06-477d9c4bd132 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:35.350621 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdf5d78-ad51-46d9-bc06-477d9c4bd132 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:35.370898 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-0cdf5d78-ad51-46d9-bc06-477d9c4bd132 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:38:35.370898 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:38:35.370898 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:38:35.370898 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:38:35.370898 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:38:35.370898 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:38:35.370898 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:38:35.370898 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:38:35.370898 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:38:35.370898 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:38:35.370898 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:38:35.370898 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:38:35.370898 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:38:35.370898 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:38:35.370898 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:38:35.371786 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:38:35.371786 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:38:35.371786 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:38:35.371786 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:38:35.371786 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:38:35.371786 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:38:35.371786 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:38:35.371786 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:38:35.371786 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:38:35.371786 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:38:35.371786 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:38:35.371786 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:38:35.371786 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:38:35.371786 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:38:35.371786 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:38:35.371786 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:38:35.371786 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:38:35.372691 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:38:35.372691 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:38:35.372691 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:38:35.372691 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:38:35.372691 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 618/1224] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:38:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:38:35.380590 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b12f8-871d-4a56-b63c-67791d21fdc7 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:35.481267 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b12f8-871d-4a56-b63c-67791d21fdc7 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:35.516250 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6761573-26b7-4d9e-bfa1-92c9438cb104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:35.516906 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6761573-26b7-4d9e-bfa1-92c9438cb104 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:35.517191 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6761573-26b7-4d9e-bfa1-92c9438cb104 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:35.517468 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6761573-26b7-4d9e-bfa1-92c9438cb104 None admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:35.518191 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6761573-26b7-4d9e-bfa1-92c9438cb104 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:38:35.518501 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6761573-26b7-4d9e-bfa1-92c9438cb104 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:35.519944 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6761573-26b7-4d9e-bfa1-92c9438cb104 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:35.531273 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 607/1225] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:38:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:38:35.537881 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdf5d78-ad51-46d9-bc06-477d9c4bd132 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:35.631983 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:35.638508 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdf5d78-ad51-46d9-bc06-477d9c4bd132 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:35.670999 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da97a7cc-5323-4039-9561-e1667984cc6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:35.671786 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97a7cc-5323-4039-9561-e1667984cc6b None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:35.672065 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97a7cc-5323-4039-9561-e1667984cc6b None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:35.672984 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97a7cc-5323-4039-9561-e1667984cc6b None admin] PATH_INFO: `/v3/users/placement` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:35.680199 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da97a7cc-5323-4039-9561-e1667984cc6b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:35.681608 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da97a7cc-5323-4039-9561-e1667984cc6b None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:35.695482 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-da97a7cc-5323-4039-9561-e1667984cc6b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 22:38:35.695482 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:38:35.695482 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:38:35.695482 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:38:35.695482 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:38:35.695482 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:38:35.695482 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:38:35.695482 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:38:35.695482 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:38:35.695482 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:38:35.695482 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:38:35.695482 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:38:35.695482 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:38:35.695482 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:38:35.695482 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:38:35.695482 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:38:35.696977 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:38:35.696977 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:38:35.696977 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:38:35.696977 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:38:35.696977 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:38:35.696977 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:38:35.696977 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:38:35.696977 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:38:35.696977 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:38:35.696977 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:38:35.696977 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:38:35.696977 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:38:35.696977 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:38:35.696977 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:38:35.696977 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:38:35.696977 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:38:35.696977 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:38:35.696977 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:38:35.698172 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:38:35.698172 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:38:35.698172 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:38:35.698172 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:38:35.698172 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:38:35.698172 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:38:35.698172 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:38:35.698172 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:38:35.698172 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 22:38:35.698172 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:38:35.698172 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 619/1226] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:38:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:38:35.702581 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6761573-26b7-4d9e-bfa1-92c9438cb104 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:35.749791 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:35.802880 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6761573-26b7-4d9e-bfa1-92c9438cb104 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:35.835935 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b236ac88-6171-4edf-8a1e-894e64af60e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:35.836600 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236ac88-6171-4edf-8a1e-894e64af60e5 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:35.836885 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236ac88-6171-4edf-8a1e-894e64af60e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:35.837159 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236ac88-6171-4edf-8a1e-894e64af60e5 None admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:35.837987 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b236ac88-6171-4edf-8a1e-894e64af60e5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:38:35.838294 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b236ac88-6171-4edf-8a1e-894e64af60e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:35.839823 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b236ac88-6171-4edf-8a1e-894e64af60e5 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:35.858330 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 608/1227] 10.222.0.52 () {56 vars in 1143 bytes} [Thu Jul 21 22:38:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:38:35.864045 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da97a7cc-5323-4039-9561-e1667984cc6b None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:35.963912 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da97a7cc-5323-4039-9561-e1667984cc6b None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:35.997382 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e968441f-0eec-4adf-a1ad-53ad4382a888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:35.998164 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e968441f-0eec-4adf-a1ad-53ad4382a888 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:35.998446 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e968441f-0eec-4adf-a1ad-53ad4382a888 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:35.998721 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e968441f-0eec-4adf-a1ad-53ad4382a888 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:36.009058 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e968441f-0eec-4adf-a1ad-53ad4382a888 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:36.010480 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e968441f-0eec-4adf-a1ad-53ad4382a888 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:36.027831 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-e968441f-0eec-4adf-a1ad-53ad4382a888 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:38:36.027831 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:38:36.027831 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:38:36.027831 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:38:36.027831 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:38:36.027831 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:38:36.027831 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:38:36.027831 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:38:36.027831 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:38:36.027831 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:38:36.027831 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:38:36.027831 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:38:36.027831 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:38:36.027831 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:38:36.027831 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:38:36.028953 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:38:36.028953 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:38:36.028953 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:38:36.028953 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:38:36.028953 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:38:36.028953 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:38:36.028953 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:38:36.028953 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:38:36.028953 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:38:36.028953 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:38:36.028953 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:38:36.028953 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:38:36.028953 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:38:36.028953 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:38:36.028953 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:38:36.028953 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:38:36.028953 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:38:36.030066 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:38:36.030066 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:38:36.030066 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:38:36.030066 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:38:36.030066 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 620/1228] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:38:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:38:36.035369 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b236ac88-6171-4edf-8a1e-894e64af60e5 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:36.136731 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b236ac88-6171-4edf-8a1e-894e64af60e5 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:36.139390 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:36.170568 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c63379-eb58-46ef-9b95-ea1d035f5bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:36.171367 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c63379-eb58-46ef-9b95-ea1d035f5bce None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:36.171647 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c63379-eb58-46ef-9b95-ea1d035f5bce None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:36.171936 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c63379-eb58-46ef-9b95-ea1d035f5bce None admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:36.173279 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c63379-eb58-46ef-9b95-ea1d035f5bce None admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:38:36.173579 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c63379-eb58-46ef-9b95-ea1d035f5bce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:36.175068 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c63379-eb58-46ef-9b95-ea1d035f5bce None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:36.186333 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 609/1229] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:38:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:38:36.192316 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e968441f-0eec-4adf-a1ad-53ad4382a888 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:36.201789 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:36.291028 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e968441f-0eec-4adf-a1ad-53ad4382a888 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:36.323617 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b035bc-4021-460b-b9c9-6a9d3cd9e956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:36.324393 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b035bc-4021-460b-b9c9-6a9d3cd9e956 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:36.324675 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b035bc-4021-460b-b9c9-6a9d3cd9e956 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:36.324948 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b035bc-4021-460b-b9c9-6a9d3cd9e956 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:36.331492 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b035bc-4021-460b-b9c9-6a9d3cd9e956 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:36.332917 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b035bc-4021-460b-b9c9-6a9d3cd9e956 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:36.347377 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-08b035bc-4021-460b-b9c9-6a9d3cd9e956 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:38:36.347377 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:38:36.347377 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:38:36.347377 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:38:36.347377 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:38:36.347377 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:38:36.347377 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:38:36.347377 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:38:36.347377 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:38:36.347377 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:38:36.347377 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:38:36.347377 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:38:36.347377 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:38:36.347377 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:38:36.347377 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:38:36.348724 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:38:36.348724 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:38:36.348724 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:38:36.348724 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:38:36.348724 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:38:36.348724 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:38:36.348724 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:38:36.348724 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:38:36.348724 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:38:36.348724 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:38:36.348724 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:38:36.348724 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:38:36.348724 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:38:36.348724 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:38:36.348724 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:38:36.348724 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:38:36.348724 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:38:36.349960 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:38:36.349960 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:38:36.349960 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:38:36.349960 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:38:36.349960 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:38:36.349960 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:38:36.349960 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:38:36.349960 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:38:36.349960 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 621/1230] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:38:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:38:36.354803 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c63379-eb58-46ef-9b95-ea1d035f5bce None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:36.457586 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c63379-eb58-46ef-9b95-ea1d035f5bce None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:36.491769 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52701708-e2fb-46cb-b6e5-d4528a6ea305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:36.492434 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52701708-e2fb-46cb-b6e5-d4528a6ea305 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:36.492718 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52701708-e2fb-46cb-b6e5-d4528a6ea305 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:36.492999 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52701708-e2fb-46cb-b6e5-d4528a6ea305 None admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:36.493784 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52701708-e2fb-46cb-b6e5-d4528a6ea305 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:38:36.494093 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52701708-e2fb-46cb-b6e5-d4528a6ea305 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:36.495555 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52701708-e2fb-46cb-b6e5-d4528a6ea305 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:36.510590 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 610/1231] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 22:38:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:38:36.513659 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b035bc-4021-460b-b9c9-6a9d3cd9e956 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:36.578642 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:36.612479 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b035bc-4021-460b-b9c9-6a9d3cd9e956 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:36.646143 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe77c330-f260-4480-9611-59605521bae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:36.647042 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77c330-f260-4480-9611-59605521bae4 None admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:36.647328 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77c330-f260-4480-9611-59605521bae4 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:36.647602 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77c330-f260-4480-9611-59605521bae4 None admin] PATH_INFO: `/v3/projects/aa11ab8229b642ce962193b74001d3bc/users/10c0bed169344a77bfad29624c5e5a8e/roles/715d866890d14923b11470d544b4f960` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:36.661413 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe77c330-f260-4480-9611-59605521bae4 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa11ab8229b642ce962193b74001d3bc, user_id=10c0bed169344a77bfad29624c5e5a8e, role_id=715d866890d14923b11470d544b4f960)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:36.662942 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe77c330-f260-4480-9611-59605521bae4 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:36.783991 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 622/1232] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 21 22:38:36 2022] PUT /identity/v3/projects/aa11ab8229b642ce962193b74001d3bc/users/10c0bed169344a77bfad29624c5e5a8e/roles/715d866890d14923b11470d544b4f960 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:38:38.801446 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38279c-bc6e-4c9e-b595-c332850a5150 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:38.801988 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38279c-bc6e-4c9e-b595-c332850a5150 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:38.801988 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38279c-bc6e-4c9e-b595-c332850a5150 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:38.803370 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 611/1233] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:38:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:38:38.808481 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2fc6a9-77d9-4720-9c5b-315523180ce1 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:38.808869 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2fc6a9-77d9-4720-9c5b-315523180ce1 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:38.809046 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2fc6a9-77d9-4720-9c5b-315523180ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:38.825618 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-fa2fc6a9-77d9-4720-9c5b-315523180ce1 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:38:38.826824 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-fa2fc6a9-77d9-4720-9c5b-315523180ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:38.893048 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 623/1234] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:38:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:38:38.916119 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab38279c-bc6e-4c9e-b595-c332850a5150 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:38.941960 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:39.019012 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab38279c-bc6e-4c9e-b595-c332850a5150 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:39.052833 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f26853-f9a6-4c0e-a2a8-e9aea3680e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:39.053551 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f26853-f9a6-4c0e-a2a8-e9aea3680e90 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:39.053829 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f26853-f9a6-4c0e-a2a8-e9aea3680e90 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:39.054101 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f26853-f9a6-4c0e-a2a8-e9aea3680e90 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:39.064696 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f26853-f9a6-4c0e-a2a8-e9aea3680e90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:38:39.073681 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-19f26853-f9a6-4c0e-a2a8-e9aea3680e90 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:38:39.073681 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:38:39.073681 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:38:39.073681 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:38:39.073681 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:38:39.073681 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:38:39.073681 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:38:39.073681 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:38:39.073681 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:38:39.073681 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:38:39.073681 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:38:39.073681 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:38:39.073681 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:38:39.073681 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:38:39.073681 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:38:39.073681 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:38:39.076309 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:38:39.076309 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:38:39.076309 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:38:39.076309 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:38:39.076309 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:38:39.076309 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:38:39.076309 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:38:39.076309 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:38:39.076309 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:38:39.076309 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:38:39.076309 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:38:39.076309 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:38:39.076309 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:38:39.076309 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:38:39.076309 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:38:39.076309 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:38:39.077255 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:38:39.077255 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:38:39.077255 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:38:39.077255 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:38:39.077255 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:38:39.077255 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:38:39.077255 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:38:39.077255 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:38:39.077255 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:38:39.077255 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:38:39.077255 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 22:38:39.077255 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:38:39.077255 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 612/1235] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Jul 21 22:38:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:38:39.083093 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2fc6a9-77d9-4720-9c5b-315523180ce1 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:39.130723 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:39.182422 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2fc6a9-77d9-4720-9c5b-315523180ce1 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:39.215529 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d21d18-a037-462d-8af3-b9e6a48cadee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:39.216205 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d21d18-a037-462d-8af3-b9e6a48cadee None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:39.216484 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d21d18-a037-462d-8af3-b9e6a48cadee None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:39.216753 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d21d18-a037-462d-8af3-b9e6a48cadee None admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:39.217511 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d21d18-a037-462d-8af3-b9e6a48cadee None admin] RBAC: Adding query filter params (name=service) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:38:39.217835 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d21d18-a037-462d-8af3-b9e6a48cadee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:39.219229 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d21d18-a037-462d-8af3-b9e6a48cadee None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:39.228368 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 624/1236] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Jul 21 22:38:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:38:39.234099 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f26853-f9a6-4c0e-a2a8-e9aea3680e90 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:39.321897 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:39.337542 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f26853-f9a6-4c0e-a2a8-e9aea3680e90 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:39.371292 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a1106-9ff2-42ec-89d0-1a3a6ddf14c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:39.372084 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a1106-9ff2-42ec-89d0-1a3a6ddf14c5 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:39.372369 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a1106-9ff2-42ec-89d0-1a3a6ddf14c5 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:39.372646 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a1106-9ff2-42ec-89d0-1a3a6ddf14c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:39.383540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a1106-9ff2-42ec-89d0-1a3a6ddf14c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:39.385032 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a1106-9ff2-42ec-89d0-1a3a6ddf14c5 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:39.402753 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-cf8a1106-9ff2-42ec-89d0-1a3a6ddf14c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:38:39.402753 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:38:39.402753 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:38:39.402753 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:38:39.402753 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:38:39.402753 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:38:39.402753 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:38:39.402753 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:38:39.402753 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:38:39.402753 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:38:39.402753 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:38:39.402753 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:38:39.402753 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:38:39.402753 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:38:39.402753 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:38:39.403825 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:38:39.403825 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:38:39.403825 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:38:39.403825 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:38:39.403825 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:38:39.403825 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:38:39.403825 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:38:39.403825 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:38:39.403825 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:38:39.403825 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:38:39.403825 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:38:39.403825 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:38:39.403825 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:38:39.403825 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:38:39.403825 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:38:39.403825 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:38:39.403825 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:38:39.405141 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:38:39.405141 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:38:39.405141 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:38:39.405141 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:38:39.405141 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 613/1237] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:38:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:38:39.409905 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d21d18-a037-462d-8af3-b9e6a48cadee None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:39.509953 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d21d18-a037-462d-8af3-b9e6a48cadee None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:39.520597 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:39.544694 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd59499c-781e-4c4f-a8d4-e90fef5ed03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:39.545361 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd59499c-781e-4c4f-a8d4-e90fef5ed03b None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:39.545646 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd59499c-781e-4c4f-a8d4-e90fef5ed03b None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:39.545921 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd59499c-781e-4c4f-a8d4-e90fef5ed03b None admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:39.546691 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd59499c-781e-4c4f-a8d4-e90fef5ed03b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:38:39.546980 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd59499c-781e-4c4f-a8d4-e90fef5ed03b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:39.548827 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd59499c-781e-4c4f-a8d4-e90fef5ed03b None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:39.559886 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 625/1238] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:38:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:38:39.566496 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a1106-9ff2-42ec-89d0-1a3a6ddf14c5 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:39.664681 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a1106-9ff2-42ec-89d0-1a3a6ddf14c5 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:39.698361 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2706149e-b47b-45fe-b1d6-dd19b9cd48b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:39.699174 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2706149e-b47b-45fe-b1d6-dd19b9cd48b1 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:39.699454 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2706149e-b47b-45fe-b1d6-dd19b9cd48b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:39.699724 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2706149e-b47b-45fe-b1d6-dd19b9cd48b1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:39.702356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:39.706728 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2706149e-b47b-45fe-b1d6-dd19b9cd48b1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:39.708209 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2706149e-b47b-45fe-b1d6-dd19b9cd48b1 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:39.722418 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-2706149e-b47b-45fe-b1d6-dd19b9cd48b1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 22:38:39.722418 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:38:39.722418 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:38:39.722418 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:38:39.722418 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:38:39.722418 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:38:39.722418 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:38:39.722418 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:38:39.722418 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:38:39.722418 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:38:39.722418 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:38:39.722418 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:38:39.722418 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:38:39.722418 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:38:39.722418 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:38:39.722418 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:38:39.725902 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:38:39.725902 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:38:39.725902 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:38:39.725902 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:38:39.725902 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:38:39.725902 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:38:39.725902 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:38:39.725902 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:38:39.725902 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:38:39.725902 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:38:39.725902 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:38:39.725902 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:38:39.725902 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:38:39.725902 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:38:39.725902 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:38:39.725902 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:38:39.725902 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:38:39.725902 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:38:39.726932 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:38:39.726932 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:38:39.726932 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:38:39.726932 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:38:39.726932 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:38:39.726932 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:38:39.726932 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:38:39.726932 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:38:39.726932 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 22:38:39.726932 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:38:39.726932 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 614/1239] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:38:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:38:39.729899 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd59499c-781e-4c4f-a8d4-e90fef5ed03b None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:39.833830 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd59499c-781e-4c4f-a8d4-e90fef5ed03b None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:39.867121 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864d184-241d-48bf-80b6-dda5ad0e2303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:39.867781 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864d184-241d-48bf-80b6-dda5ad0e2303 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:39.868092 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864d184-241d-48bf-80b6-dda5ad0e2303 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:39.868366 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864d184-241d-48bf-80b6-dda5ad0e2303 None admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:39.869185 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a864d184-241d-48bf-80b6-dda5ad0e2303 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:38:39.869534 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a864d184-241d-48bf-80b6-dda5ad0e2303 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:39.870997 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a864d184-241d-48bf-80b6-dda5ad0e2303 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:39.888347 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 626/1240] 10.222.0.52 () {56 vars in 1143 bytes} [Thu Jul 21 22:38:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:38:39.894353 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2706149e-b47b-45fe-b1d6-dd19b9cd48b1 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:39.993384 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2706149e-b47b-45fe-b1d6-dd19b9cd48b1 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:40.026013 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b874e1b-bdea-4075-a0ae-4241e7beabd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:40.026676 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b874e1b-bdea-4075-a0ae-4241e7beabd4 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:40.026969 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b874e1b-bdea-4075-a0ae-4241e7beabd4 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:40.027246 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b874e1b-bdea-4075-a0ae-4241e7beabd4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:40.086920 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:40.093257 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b874e1b-bdea-4075-a0ae-4241e7beabd4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:40.094691 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b874e1b-bdea-4075-a0ae-4241e7beabd4 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:40.194359 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 615/1241] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:38:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 22:38:40.200198 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864d184-241d-48bf-80b6-dda5ad0e2303 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:40.227996 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:40.298851 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864d184-241d-48bf-80b6-dda5ad0e2303 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:40.331788 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5c1a8f-b248-43b4-b8c7-635aa291ab4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:40.332580 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c1a8f-b248-43b4-b8c7-635aa291ab4d None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:40.332865 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c1a8f-b248-43b4-b8c7-635aa291ab4d None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:40.333144 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c1a8f-b248-43b4-b8c7-635aa291ab4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:40.342923 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5c1a8f-b248-43b4-b8c7-635aa291ab4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:40.344365 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5c1a8f-b248-43b4-b8c7-635aa291ab4d None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:40.362204 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-9d5c1a8f-b248-43b4-b8c7-635aa291ab4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:38:40.362204 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:38:40.362204 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:38:40.362204 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:38:40.362204 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:38:40.362204 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:38:40.362204 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:38:40.362204 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:38:40.362204 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:38:40.362204 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:38:40.362204 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:38:40.362204 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:38:40.362204 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:38:40.362204 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:38:40.362204 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:38:40.363281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:38:40.363281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:38:40.363281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:38:40.363281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:38:40.363281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:38:40.363281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:38:40.363281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:38:40.363281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:38:40.363281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:38:40.363281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:38:40.363281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:38:40.363281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:38:40.363281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:38:40.363281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:38:40.363281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:38:40.363281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:38:40.363281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:38:40.369239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:38:40.369239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:38:40.369239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:38:40.369239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:38:40.369239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 627/1242] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:38:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:38:40.369614 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b874e1b-bdea-4075-a0ae-4241e7beabd4 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:40.465695 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:40.471995 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b874e1b-bdea-4075-a0ae-4241e7beabd4 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:40.505120 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f2d88b-d2e4-4bb1-a8c4-7c4b3d7d74aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:40.505780 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2d88b-d2e4-4bb1-a8c4-7c4b3d7d74aa None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:40.506060 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2d88b-d2e4-4bb1-a8c4-7c4b3d7d74aa None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:40.506333 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2d88b-d2e4-4bb1-a8c4-7c4b3d7d74aa None admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:40.507097 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f2d88b-d2e4-4bb1-a8c4-7c4b3d7d74aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:38:40.507401 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f2d88b-d2e4-4bb1-a8c4-7c4b3d7d74aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:40.508839 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f2d88b-d2e4-4bb1-a8c4-7c4b3d7d74aa None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:40.520317 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 616/1243] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:38:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:38:40.526263 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5c1a8f-b248-43b4-b8c7-635aa291ab4d None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:40.604028 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:40.625237 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5c1a8f-b248-43b4-b8c7-635aa291ab4d None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:40.661069 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c37292-3cef-46a8-942b-758faa750416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:40.661875 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c37292-3cef-46a8-942b-758faa750416 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:40.662164 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c37292-3cef-46a8-942b-758faa750416 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:40.662446 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c37292-3cef-46a8-942b-758faa750416 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:40.669452 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c37292-3cef-46a8-942b-758faa750416 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:40.670924 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c37292-3cef-46a8-942b-758faa750416 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:40.684974 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-72c37292-3cef-46a8-942b-758faa750416 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:38:40.684974 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:38:40.684974 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:38:40.684974 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:38:40.684974 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:38:40.684974 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:38:40.684974 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:38:40.684974 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:38:40.684974 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:38:40.684974 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:38:40.684974 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:38:40.684974 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:38:40.684974 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:38:40.684974 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:38:40.684974 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:38:40.695851 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:38:40.695851 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:38:40.695851 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:38:40.695851 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:38:40.695851 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:38:40.695851 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:38:40.695851 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:38:40.695851 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:38:40.695851 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:38:40.695851 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:38:40.695851 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:38:40.695851 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:38:40.695851 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:38:40.695851 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:38:40.695851 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:38:40.695851 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:38:40.695851 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:38:40.696960 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:38:40.696960 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:38:40.696960 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:38:40.696960 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:38:40.696960 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:38:40.696960 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:38:40.696960 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:38:40.696960 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:38:40.696960 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 628/1244] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:38:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:38:40.697588 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f2d88b-d2e4-4bb1-a8c4-7c4b3d7d74aa None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:40.790718 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f2d88b-d2e4-4bb1-a8c4-7c4b3d7d74aa None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:40.823759 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e66d31-1f09-4af1-ae3d-991bbc9fba77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:40.824413 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e66d31-1f09-4af1-ae3d-991bbc9fba77 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:40.824694 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e66d31-1f09-4af1-ae3d-991bbc9fba77 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:40.824970 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e66d31-1f09-4af1-ae3d-991bbc9fba77 None admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:40.825754 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e66d31-1f09-4af1-ae3d-991bbc9fba77 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:38:40.826066 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e66d31-1f09-4af1-ae3d-991bbc9fba77 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:40.827523 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e66d31-1f09-4af1-ae3d-991bbc9fba77 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:40.839264 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 617/1245] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 22:38:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:38:40.845963 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c37292-3cef-46a8-942b-758faa750416 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:40.947221 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c37292-3cef-46a8-942b-758faa750416 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:40.980860 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc0446e-f046-46a8-bab8-78f97f15a9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:40.981512 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc0446e-f046-46a8-bab8-78f97f15a9aa None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:40.981798 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc0446e-f046-46a8-bab8-78f97f15a9aa None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:40.982122 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc0446e-f046-46a8-bab8-78f97f15a9aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:40.982981 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc0446e-f046-46a8-bab8-78f97f15a9aa None admin] RBAC: Adding query filter params (role.id=715d866890d14923b11470d544b4f960, scope.project.id=aa11ab8229b642ce962193b74001d3bc, user.id=10c0bed169344a77bfad29624c5e5a8e) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:38:40.983299 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc0446e-f046-46a8-bab8-78f97f15a9aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:40.984757 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc0446e-f046-46a8-bab8-78f97f15a9aa None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:40.991345 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 629/1246] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 22:38:40 2022] GET /identity/v3/role_assignments?user.id=10c0bed169344a77bfad29624c5e5a8e&scope.project.id=aa11ab8229b642ce962193b74001d3bc&role.id=715d866890d14923b11470d544b4f960 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:38:43.089882 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5824da8-9bf6-4c96-8340-31abad0dc398 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:43.090403 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5824da8-9bf6-4c96-8340-31abad0dc398 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:43.090403 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5824da8-9bf6-4c96-8340-31abad0dc398 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:43.091844 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 618/1247] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:38:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:38:43.097697 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a40469-5449-4e0a-bd52-fa53841bdc0a None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:43.098574 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a40469-5449-4e0a-bd52-fa53841bdc0a None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:43.098574 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a40469-5449-4e0a-bd52-fa53841bdc0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:43.103642 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:43.116349 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-f4a40469-5449-4e0a-bd52-fa53841bdc0a None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:38:43.117544 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-f4a40469-5449-4e0a-bd52-fa53841bdc0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:43.188756 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 630/1248] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:38:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 22:38:43.212269 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5824da8-9bf6-4c96-8340-31abad0dc398 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:43.231289 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:43.313517 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5824da8-9bf6-4c96-8340-31abad0dc398 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:43.348754 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b068bef0-4d18-49eb-b73b-643136bf4c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:43.349545 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b068bef0-4d18-49eb-b73b-643136bf4c90 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:43.349778 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b068bef0-4d18-49eb-b73b-643136bf4c90 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:43.350053 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b068bef0-4d18-49eb-b73b-643136bf4c90 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:43.360093 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b068bef0-4d18-49eb-b73b-643136bf4c90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:38:43.367690 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-b068bef0-4d18-49eb-b73b-643136bf4c90 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:38:43.367690 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:38:43.367690 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:38:43.367690 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:38:43.367690 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:38:43.367690 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:38:43.367690 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:38:43.367690 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:38:43.367690 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:38:43.367690 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:38:43.367690 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:38:43.367690 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:38:43.367690 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:38:43.367690 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:38:43.367690 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:38:43.367690 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:38:43.368792 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:38:43.368792 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:38:43.368792 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:38:43.368792 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:38:43.368792 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:38:43.368792 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:38:43.368792 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:38:43.368792 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:38:43.368792 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:38:43.368792 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:38:43.368792 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:38:43.368792 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:38:43.368792 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:38:43.368792 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:38:43.368792 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:38:43.368792 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:38:43.369951 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:38:43.369951 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:38:43.369951 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:38:43.369951 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:38:43.369951 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:38:43.369951 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:38:43.369951 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:38:43.369951 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:38:43.369951 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:38:43.369951 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:38:43.369951 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:38:43.369951 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:38:43.369951 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 619/1249] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 22:38:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:38:43.374986 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a40469-5449-4e0a-bd52-fa53841bdc0a None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:43.478629 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a40469-5449-4e0a-bd52-fa53841bdc0a None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:43.510633 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:43.515566 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe3eb0-606c-4987-ac2e-f98964c67e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:43.516238 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe3eb0-606c-4987-ac2e-f98964c67e34 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:43.516529 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe3eb0-606c-4987-ac2e-f98964c67e34 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:43.516823 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe3eb0-606c-4987-ac2e-f98964c67e34 None admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:43.518047 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefe3eb0-606c-4987-ac2e-f98964c67e34 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:38:43.518351 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefe3eb0-606c-4987-ac2e-f98964c67e34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:43.519804 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefe3eb0-606c-4987-ac2e-f98964c67e34 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:43.527709 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 631/1250] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:38:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:38:43.534030 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b068bef0-4d18-49eb-b73b-643136bf4c90 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:43.613339 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:43.636253 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b068bef0-4d18-49eb-b73b-643136bf4c90 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:43.668981 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae6a0c-a4aa-4a8e-9cc8-70ec835b2a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:43.670004 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae6a0c-a4aa-4a8e-9cc8-70ec835b2a40 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:43.670476 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae6a0c-a4aa-4a8e-9cc8-70ec835b2a40 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:43.671227 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae6a0c-a4aa-4a8e-9cc8-70ec835b2a40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:43.682346 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae6a0c-a4aa-4a8e-9cc8-70ec835b2a40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:43.683800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae6a0c-a4aa-4a8e-9cc8-70ec835b2a40 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:43.701542 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-71ae6a0c-a4aa-4a8e-9cc8-70ec835b2a40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:38:43.701542 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:38:43.701542 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:38:43.701542 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:38:43.701542 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:38:43.701542 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:38:43.701542 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:38:43.701542 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:38:43.701542 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:38:43.701542 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:38:43.701542 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:38:43.701542 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:38:43.701542 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:38:43.701542 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:38:43.701542 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:38:43.702684 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:38:43.702684 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:38:43.702684 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:38:43.702684 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:38:43.702684 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:38:43.702684 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:38:43.702684 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:38:43.702684 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:38:43.702684 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:38:43.702684 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:38:43.702684 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:38:43.702684 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:38:43.702684 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:38:43.702684 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:38:43.702684 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:38:43.702684 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:38:43.702684 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:38:43.703691 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:38:43.703691 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:38:43.703691 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:38:43.703691 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:38:43.703691 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 620/1251] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:38:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:38:43.708893 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe3eb0-606c-4987-ac2e-f98964c67e34 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:43.812420 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe3eb0-606c-4987-ac2e-f98964c67e34 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:43.846993 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d98a4e-cc13-4c3f-bb93-a715fcec89a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:43.847686 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d98a4e-cc13-4c3f-bb93-a715fcec89a5 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:43.848225 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d98a4e-cc13-4c3f-bb93-a715fcec89a5 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:43.848508 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d98a4e-cc13-4c3f-bb93-a715fcec89a5 None admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:43.849224 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d98a4e-cc13-4c3f-bb93-a715fcec89a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:38:43.849531 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d98a4e-cc13-4c3f-bb93-a715fcec89a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:43.850997 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d98a4e-cc13-4c3f-bb93-a715fcec89a5 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:43.863127 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 632/1252] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:38:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:38:43.868880 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae6a0c-a4aa-4a8e-9cc8-70ec835b2a40 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:43.967341 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae6a0c-a4aa-4a8e-9cc8-70ec835b2a40 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:43.994326 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:44.001967 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a89fe85-189e-497d-b9ca-e4b585ea97ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:44.002765 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a89fe85-189e-497d-b9ca-e4b585ea97ed None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:44.003075 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a89fe85-189e-497d-b9ca-e4b585ea97ed None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:44.003351 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a89fe85-189e-497d-b9ca-e4b585ea97ed None admin] PATH_INFO: `/v3/users/placement` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:44.011420 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a89fe85-189e-497d-b9ca-e4b585ea97ed None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:44.013683 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a89fe85-189e-497d-b9ca-e4b585ea97ed None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:44.028282 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-7a89fe85-189e-497d-b9ca-e4b585ea97ed None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 22:38:44.028282 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:38:44.028282 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:38:44.028282 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:38:44.028282 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:38:44.028282 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:38:44.028282 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:38:44.028282 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:38:44.028282 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:38:44.028282 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:38:44.028282 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:38:44.028282 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:38:44.028282 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:38:44.028282 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:38:44.028282 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:38:44.028282 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:38:44.029441 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:38:44.029441 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:38:44.029441 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:38:44.029441 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:38:44.029441 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:38:44.029441 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:38:44.029441 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:38:44.029441 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:38:44.029441 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:38:44.029441 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:38:44.029441 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:38:44.029441 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:38:44.029441 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:38:44.029441 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:38:44.029441 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:38:44.029441 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:38:44.029441 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:38:44.029441 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:38:44.030432 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:38:44.030432 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:38:44.030432 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:38:44.030432 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:38:44.030432 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:38:44.030432 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:38:44.030432 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:38:44.030432 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:38:44.030432 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 22:38:44.030432 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:38:44.030432 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 621/1253] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:38:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:38:44.036089 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d98a4e-cc13-4c3f-bb93-a715fcec89a5 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:44.091801 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:44.136604 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d98a4e-cc13-4c3f-bb93-a715fcec89a5 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:44.169567 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a055f0-cf1a-4f5d-a08e-9aa5e5100311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:44.170226 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a055f0-cf1a-4f5d-a08e-9aa5e5100311 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:44.170509 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a055f0-cf1a-4f5d-a08e-9aa5e5100311 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:44.170828 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a055f0-cf1a-4f5d-a08e-9aa5e5100311 None admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:44.171720 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a055f0-cf1a-4f5d-a08e-9aa5e5100311 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:38:44.172029 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a055f0-cf1a-4f5d-a08e-9aa5e5100311 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:44.173963 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a055f0-cf1a-4f5d-a08e-9aa5e5100311 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:44.191474 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 633/1254] 10.222.0.52 () {56 vars in 1143 bytes} [Thu Jul 21 22:38:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:38:44.197388 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a89fe85-189e-497d-b9ca-e4b585ea97ed None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:44.295228 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a89fe85-189e-497d-b9ca-e4b585ea97ed None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:44.329794 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69318eb8-441e-42cd-ae96-3e1a974ab752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:44.330438 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69318eb8-441e-42cd-ae96-3e1a974ab752 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:44.330720 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69318eb8-441e-42cd-ae96-3e1a974ab752 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:44.331059 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69318eb8-441e-42cd-ae96-3e1a974ab752 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:44.386964 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:44.398988 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69318eb8-441e-42cd-ae96-3e1a974ab752 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:44.400452 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69318eb8-441e-42cd-ae96-3e1a974ab752 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:44.500575 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 622/1255] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:38:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 22:38:44.507015 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a055f0-cf1a-4f5d-a08e-9aa5e5100311 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:44.606137 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a055f0-cf1a-4f5d-a08e-9aa5e5100311 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:44.619858 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:44.640623 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab522196-f21c-4e8c-92f4-7d8106287d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:44.641355 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab522196-f21c-4e8c-92f4-7d8106287d4e None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:44.641641 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab522196-f21c-4e8c-92f4-7d8106287d4e None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:44.641919 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab522196-f21c-4e8c-92f4-7d8106287d4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:44.652270 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab522196-f21c-4e8c-92f4-7d8106287d4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:44.653698 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab522196-f21c-4e8c-92f4-7d8106287d4e None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:44.671164 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-ab522196-f21c-4e8c-92f4-7d8106287d4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:38:44.671164 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:38:44.671164 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:38:44.671164 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:38:44.671164 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:38:44.671164 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:38:44.671164 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:38:44.671164 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:38:44.671164 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:38:44.671164 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:38:44.671164 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:38:44.671164 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:38:44.671164 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:38:44.671164 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:38:44.671164 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:38:44.681092 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:38:44.681092 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:38:44.681092 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:38:44.681092 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:38:44.681092 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:38:44.681092 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:38:44.681092 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:38:44.681092 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:38:44.681092 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:38:44.681092 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:38:44.681092 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:38:44.681092 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:38:44.681092 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:38:44.681092 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:38:44.681092 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:38:44.681092 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:38:44.681092 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:38:44.682036 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:38:44.682036 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:38:44.682036 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:38:44.682036 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:38:44.682036 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 634/1256] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:38:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:38:44.682323 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69318eb8-441e-42cd-ae96-3e1a974ab752 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:44.764819 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:44.776192 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69318eb8-441e-42cd-ae96-3e1a974ab752 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:44.809770 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5471acf1-9f96-43da-bc01-357c5cb48c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:44.810422 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471acf1-9f96-43da-bc01-357c5cb48c99 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:44.810701 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471acf1-9f96-43da-bc01-357c5cb48c99 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:44.811007 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471acf1-9f96-43da-bc01-357c5cb48c99 None admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:44.811752 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5471acf1-9f96-43da-bc01-357c5cb48c99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:38:44.812053 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5471acf1-9f96-43da-bc01-357c5cb48c99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:44.813396 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5471acf1-9f96-43da-bc01-357c5cb48c99 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:44.825553 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 623/1257] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:38:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:38:44.831604 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab522196-f21c-4e8c-92f4-7d8106287d4e None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:44.927944 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab522196-f21c-4e8c-92f4-7d8106287d4e None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:44.960595 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b704a5e4-6094-4006-b479-2e16d9a9fb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:44.961366 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b704a5e4-6094-4006-b479-2e16d9a9fb94 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:44.961642 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b704a5e4-6094-4006-b479-2e16d9a9fb94 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:44.961911 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b704a5e4-6094-4006-b479-2e16d9a9fb94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:44.968360 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b704a5e4-6094-4006-b479-2e16d9a9fb94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:44.969771 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b704a5e4-6094-4006-b479-2e16d9a9fb94 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:44.983408 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-b704a5e4-6094-4006-b479-2e16d9a9fb94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:38:44.983408 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:38:44.983408 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:38:44.983408 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:38:44.983408 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:38:44.983408 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:38:44.983408 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:38:44.983408 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:38:44.983408 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:38:44.983408 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:38:44.983408 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:38:44.983408 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:38:44.983408 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:38:44.983408 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:38:44.983408 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:38:44.984485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:38:44.984485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:38:44.984485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:38:44.984485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:38:44.984485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:38:44.984485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:38:44.984485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:38:44.984485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:38:44.984485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:38:44.984485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:38:44.984485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:38:44.984485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:38:44.984485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:38:44.984485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:38:44.984485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:38:44.984485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:38:44.984485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:38:44.985597 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:38:44.985597 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:38:44.985597 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:38:44.985597 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:38:44.985597 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:38:44.985597 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:38:44.985597 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:38:44.985597 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:38:44.985597 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 635/1258] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:38:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:38:44.990959 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5471acf1-9f96-43da-bc01-357c5cb48c99 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:45.088260 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5471acf1-9f96-43da-bc01-357c5cb48c99 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:45.127198 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ad6c32-db2b-4975-8547-6d3b13e590d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:45.127867 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad6c32-db2b-4975-8547-6d3b13e590d5 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:45.128162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad6c32-db2b-4975-8547-6d3b13e590d5 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:45.128445 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad6c32-db2b-4975-8547-6d3b13e590d5 None admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:45.129210 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ad6c32-db2b-4975-8547-6d3b13e590d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:38:45.129520 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ad6c32-db2b-4975-8547-6d3b13e590d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:45.130982 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ad6c32-db2b-4975-8547-6d3b13e590d5 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:45.142980 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 624/1259] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 22:38:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:38:45.149102 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b704a5e4-6094-4006-b479-2e16d9a9fb94 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:45.175911 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:45.249550 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b704a5e4-6094-4006-b479-2e16d9a9fb94 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:45.281717 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c196864c-f61c-486d-a474-e2311f5531ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:45.282359 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c196864c-f61c-486d-a474-e2311f5531ff None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:45.282643 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c196864c-f61c-486d-a474-e2311f5531ff None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:45.283457 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c196864c-f61c-486d-a474-e2311f5531ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:45.284285 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c196864c-f61c-486d-a474-e2311f5531ff None admin] RBAC: Adding query filter params (role.id=d1ddee590a964ff1bbb4ee8716173021, scope.project.id=aa11ab8229b642ce962193b74001d3bc, user.id=10c0bed169344a77bfad29624c5e5a8e) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:38:45.284597 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c196864c-f61c-486d-a474-e2311f5531ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:45.286022 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c196864c-f61c-486d-a474-e2311f5531ff None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:45.291402 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 636/1260] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 22:38:45 2022] GET /identity/v3/role_assignments?user.id=10c0bed169344a77bfad29624c5e5a8e&scope.project.id=aa11ab8229b642ce962193b74001d3bc&role.id=d1ddee590a964ff1bbb4ee8716173021 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:38:47.289704 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3488ba-1d76-4d4e-8223-1e331fa3de6c None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:47.290256 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3488ba-1d76-4d4e-8223-1e331fa3de6c None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:47.290256 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3488ba-1d76-4d4e-8223-1e331fa3de6c None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:47.291735 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 625/1261] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:38:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:38:47.296833 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc108d7a-336d-4f1d-8bb9-2ad07bc09c42 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:47.297260 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc108d7a-336d-4f1d-8bb9-2ad07bc09c42 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:47.297391 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc108d7a-336d-4f1d-8bb9-2ad07bc09c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:47.314996 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-fc108d7a-336d-4f1d-8bb9-2ad07bc09c42 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:38:47.319603 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-fc108d7a-336d-4f1d-8bb9-2ad07bc09c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:47.382203 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 637/1262] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:38:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:38:47.404548 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3488ba-1d76-4d4e-8223-1e331fa3de6c None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:47.415867 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:47.509702 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3488ba-1d76-4d4e-8223-1e331fa3de6c None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:47.546160 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56991ed-bf23-4df2-82f4-7fa69a6f10a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:47.546900 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56991ed-bf23-4df2-82f4-7fa69a6f10a7 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:47.547204 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56991ed-bf23-4df2-82f4-7fa69a6f10a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:47.547540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56991ed-bf23-4df2-82f4-7fa69a6f10a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:47.558948 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56991ed-bf23-4df2-82f4-7fa69a6f10a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:38:47.566576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-a56991ed-bf23-4df2-82f4-7fa69a6f10a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:38:47.566576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:38:47.566576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:38:47.566576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:38:47.566576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:38:47.566576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:38:47.566576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:38:47.566576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:38:47.566576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:38:47.566576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:38:47.566576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:38:47.566576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:38:47.566576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:38:47.566576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:38:47.566576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:38:47.566576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:38:47.567779 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:38:47.567779 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:38:47.567779 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:38:47.567779 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:38:47.567779 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:38:47.567779 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:38:47.567779 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:38:47.567779 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:38:47.567779 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:38:47.567779 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:38:47.567779 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:38:47.567779 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:38:47.567779 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:38:47.567779 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:38:47.567779 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:38:47.567779 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:38:47.568842 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:38:47.568842 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:38:47.568842 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:38:47.568842 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:38:47.568842 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:38:47.568842 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:38:47.568842 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:38:47.568842 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:38:47.568842 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:38:47.568842 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:38:47.568842 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:38:47.568842 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:38:47.568842 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 626/1263] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 22:38:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:38:47.573960 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc108d7a-336d-4f1d-8bb9-2ad07bc09c42 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:47.592313 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:47.673281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc108d7a-336d-4f1d-8bb9-2ad07bc09c42 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:47.706649 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3263bd8-ce61-4420-89e6-1ff1fd676180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:47.707340 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3263bd8-ce61-4420-89e6-1ff1fd676180 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:47.707623 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3263bd8-ce61-4420-89e6-1ff1fd676180 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:47.707900 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3263bd8-ce61-4420-89e6-1ff1fd676180 None admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:47.708643 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3263bd8-ce61-4420-89e6-1ff1fd676180 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:38:47.708950 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3263bd8-ce61-4420-89e6-1ff1fd676180 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:47.710294 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3263bd8-ce61-4420-89e6-1ff1fd676180 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:47.718296 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 638/1264] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:38:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:38:47.725147 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56991ed-bf23-4df2-82f4-7fa69a6f10a7 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:47.791175 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:47.826072 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56991ed-bf23-4df2-82f4-7fa69a6f10a7 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:47.860944 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac1af90-f54b-4170-ae2f-b6c15b1501cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:47.861732 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1af90-f54b-4170-ae2f-b6c15b1501cc None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:47.862017 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1af90-f54b-4170-ae2f-b6c15b1501cc None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:47.862295 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1af90-f54b-4170-ae2f-b6c15b1501cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:47.873702 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac1af90-f54b-4170-ae2f-b6c15b1501cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:47.878973 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac1af90-f54b-4170-ae2f-b6c15b1501cc None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:47.892862 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-cac1af90-f54b-4170-ae2f-b6c15b1501cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:38:47.892862 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:38:47.892862 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:38:47.892862 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:38:47.892862 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:38:47.892862 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:38:47.892862 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:38:47.892862 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:38:47.892862 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:38:47.892862 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:38:47.892862 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:38:47.892862 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:38:47.892862 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:38:47.892862 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:38:47.892862 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:38:47.893911 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:38:47.893911 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:38:47.893911 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:38:47.893911 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:38:47.893911 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:38:47.893911 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:38:47.893911 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:38:47.893911 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:38:47.893911 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:38:47.893911 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:38:47.893911 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:38:47.893911 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:38:47.893911 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:38:47.893911 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:38:47.893911 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:38:47.893911 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:38:47.893911 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:38:47.895042 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:38:47.895042 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:38:47.895042 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:38:47.895042 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:38:47.895042 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 627/1265] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:38:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:38:47.901231 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3263bd8-ce61-4420-89e6-1ff1fd676180 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:47.988512 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:48.006864 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3263bd8-ce61-4420-89e6-1ff1fd676180 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:48.040867 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e759dc5-8e68-4f68-9069-eeac3dadb8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:48.041535 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e759dc5-8e68-4f68-9069-eeac3dadb8c7 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:48.041800 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e759dc5-8e68-4f68-9069-eeac3dadb8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:48.042074 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e759dc5-8e68-4f68-9069-eeac3dadb8c7 None admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:48.042807 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e759dc5-8e68-4f68-9069-eeac3dadb8c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:38:48.043117 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e759dc5-8e68-4f68-9069-eeac3dadb8c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:48.044462 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e759dc5-8e68-4f68-9069-eeac3dadb8c7 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:48.057155 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 639/1266] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:38:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:38:48.062998 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac1af90-f54b-4170-ae2f-b6c15b1501cc None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:48.163408 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac1af90-f54b-4170-ae2f-b6c15b1501cc None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:48.184018 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:48.197866 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c80ff-dd6a-45b0-8a74-bd045b6b2da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:48.198658 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c80ff-dd6a-45b0-8a74-bd045b6b2da6 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:48.198980 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c80ff-dd6a-45b0-8a74-bd045b6b2da6 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:48.199257 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c80ff-dd6a-45b0-8a74-bd045b6b2da6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:48.206511 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c80ff-dd6a-45b0-8a74-bd045b6b2da6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:48.207950 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c80ff-dd6a-45b0-8a74-bd045b6b2da6 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:48.222264 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-5e6c80ff-dd6a-45b0-8a74-bd045b6b2da6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 22:38:48.222264 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:38:48.222264 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:38:48.222264 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:38:48.222264 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:38:48.222264 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:38:48.222264 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:38:48.222264 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:38:48.222264 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:38:48.222264 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:38:48.222264 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:38:48.222264 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:38:48.222264 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:38:48.222264 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:38:48.222264 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:38:48.222264 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:38:48.223711 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:38:48.223711 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:38:48.223711 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:38:48.223711 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:38:48.223711 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:38:48.223711 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:38:48.223711 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:38:48.223711 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:38:48.223711 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:38:48.223711 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:38:48.223711 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:38:48.223711 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:38:48.223711 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:38:48.223711 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:38:48.223711 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:38:48.223711 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:38:48.223711 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:38:48.223711 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:38:48.224875 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:38:48.224875 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:38:48.224875 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:38:48.224875 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:38:48.224875 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:38:48.224875 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:38:48.224875 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:38:48.224875 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:38:48.224875 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 22:38:48.224875 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:38:48.224875 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 628/1267] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:38:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:38:48.229686 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e759dc5-8e68-4f68-9069-eeac3dadb8c7 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:48.328851 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e759dc5-8e68-4f68-9069-eeac3dadb8c7 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:48.363014 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df0f48-1aeb-438b-aa41-334ea7851fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:48.363702 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df0f48-1aeb-438b-aa41-334ea7851fcb None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:48.363977 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df0f48-1aeb-438b-aa41-334ea7851fcb None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:48.364248 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df0f48-1aeb-438b-aa41-334ea7851fcb None admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:48.365063 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49df0f48-1aeb-438b-aa41-334ea7851fcb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:38:48.365367 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49df0f48-1aeb-438b-aa41-334ea7851fcb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:48.366805 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49df0f48-1aeb-438b-aa41-334ea7851fcb None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:48.375264 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:48.383242 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 640/1268] 10.222.0.52 () {56 vars in 1143 bytes} [Thu Jul 21 22:38:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:38:48.388917 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c80ff-dd6a-45b0-8a74-bd045b6b2da6 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:48.488970 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c80ff-dd6a-45b0-8a74-bd045b6b2da6 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:48.522094 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a68536-dd04-45d0-ba33-a10fd24bf69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:48.522911 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a68536-dd04-45d0-ba33-a10fd24bf69f None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:48.523194 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a68536-dd04-45d0-ba33-a10fd24bf69f None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:48.523466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a68536-dd04-45d0-ba33-a10fd24bf69f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:48.534461 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a68536-dd04-45d0-ba33-a10fd24bf69f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:48.535923 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a68536-dd04-45d0-ba33-a10fd24bf69f None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:48.553170 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-e2a68536-dd04-45d0-ba33-a10fd24bf69f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:38:48.553170 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:38:48.553170 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:38:48.553170 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:38:48.553170 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:38:48.553170 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:38:48.553170 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:38:48.553170 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:38:48.553170 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:38:48.553170 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:38:48.553170 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:38:48.553170 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:38:48.553170 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:38:48.553170 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:38:48.553170 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:38:48.554153 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:38:48.554153 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:38:48.554153 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:38:48.554153 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:38:48.554153 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:38:48.554153 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:38:48.554153 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:38:48.554153 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:38:48.554153 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:38:48.554153 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:38:48.554153 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:38:48.554153 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:38:48.554153 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:38:48.554153 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:38:48.554153 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:38:48.554153 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:38:48.554153 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:38:48.555211 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:38:48.555211 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:38:48.555211 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:38:48.555211 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:38:48.555211 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 629/1269] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:38:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:38:48.560226 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df0f48-1aeb-438b-aa41-334ea7851fcb None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:48.659583 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df0f48-1aeb-438b-aa41-334ea7851fcb None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:48.693360 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a43dd2-bb2a-4090-a886-80042034a98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:48.694024 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a43dd2-bb2a-4090-a886-80042034a98a None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:48.694313 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a43dd2-bb2a-4090-a886-80042034a98a None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:48.694592 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a43dd2-bb2a-4090-a886-80042034a98a None admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:48.695342 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a43dd2-bb2a-4090-a886-80042034a98a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:38:48.695653 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a43dd2-bb2a-4090-a886-80042034a98a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:48.697066 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a43dd2-bb2a-4090-a886-80042034a98a None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:48.708471 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 641/1270] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:38:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:38:48.714202 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a68536-dd04-45d0-ba33-a10fd24bf69f None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:48.745954 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:48.812568 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a68536-dd04-45d0-ba33-a10fd24bf69f None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:48.847451 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c06e8ec-06ee-4cfe-b1e4-7d5e3b331083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:48.848493 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06e8ec-06ee-4cfe-b1e4-7d5e3b331083 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:48.848781 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06e8ec-06ee-4cfe-b1e4-7d5e3b331083 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:48.849055 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06e8ec-06ee-4cfe-b1e4-7d5e3b331083 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:48.855629 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c06e8ec-06ee-4cfe-b1e4-7d5e3b331083 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:48.857140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c06e8ec-06ee-4cfe-b1e4-7d5e3b331083 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:48.871203 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-0c06e8ec-06ee-4cfe-b1e4-7d5e3b331083 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:38:48.871203 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:38:48.871203 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:38:48.871203 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:38:48.871203 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:38:48.871203 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:38:48.871203 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:38:48.871203 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:38:48.871203 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:38:48.871203 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:38:48.871203 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:38:48.871203 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:38:48.871203 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:38:48.871203 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:38:48.871203 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:38:48.872250 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:38:48.872250 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:38:48.872250 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:38:48.872250 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:38:48.872250 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:38:48.872250 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:38:48.872250 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:38:48.872250 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:38:48.872250 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:38:48.872250 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:38:48.872250 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:38:48.872250 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:38:48.872250 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:38:48.872250 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:38:48.872250 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:38:48.872250 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:38:48.872250 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:38:48.873552 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:38:48.873552 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:38:48.873552 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:38:48.873552 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:38:48.873552 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:38:48.873552 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:38:48.873552 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:38:48.873552 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:38:48.873552 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 630/1271] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:38:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:38:48.878912 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a43dd2-bb2a-4090-a886-80042034a98a None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:48.939290 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:48.981044 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a43dd2-bb2a-4090-a886-80042034a98a None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:49.014058 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fea133-c43a-419d-b76c-f7ec9f3d5585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:49.014652 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fea133-c43a-419d-b76c-f7ec9f3d5585 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:49.014968 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fea133-c43a-419d-b76c-f7ec9f3d5585 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:49.015246 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fea133-c43a-419d-b76c-f7ec9f3d5585 None admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:49.016030 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fea133-c43a-419d-b76c-f7ec9f3d5585 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:38:49.016350 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fea133-c43a-419d-b76c-f7ec9f3d5585 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:49.017771 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fea133-c43a-419d-b76c-f7ec9f3d5585 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:49.029498 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 642/1272] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 22:38:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:38:49.036054 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c06e8ec-06ee-4cfe-b1e4-7d5e3b331083 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:49.129067 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:49.137860 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c06e8ec-06ee-4cfe-b1e4-7d5e3b331083 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:49.174847 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a3c56-05d6-4f64-9e23-85b55bd8364c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:49.175734 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a3c56-05d6-4f64-9e23-85b55bd8364c None admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:49.176021 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a3c56-05d6-4f64-9e23-85b55bd8364c None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:49.176299 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a3c56-05d6-4f64-9e23-85b55bd8364c None admin] PATH_INFO: `/v3/projects/aa11ab8229b642ce962193b74001d3bc/users/10c0bed169344a77bfad29624c5e5a8e/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:49.178300 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946a3c56-05d6-4f64-9e23-85b55bd8364c None admin] RBAC: Authorizing `identity:create_grant(project_id=aa11ab8229b642ce962193b74001d3bc, user_id=10c0bed169344a77bfad29624c5e5a8e, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:49.179739 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946a3c56-05d6-4f64-9e23-85b55bd8364c None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:49.343665 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 631/1273] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Jul 21 22:38:49 2022] PUT /identity/v3/projects/aa11ab8229b642ce962193b74001d3bc/users/10c0bed169344a77bfad29624c5e5a8e/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:38:51.347990 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ff2d6-b5d4-4b8a-a7af-0aace85503c1 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:51.348879 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ff2d6-b5d4-4b8a-a7af-0aace85503c1 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:51.348879 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ff2d6-b5d4-4b8a-a7af-0aace85503c1 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:51.350275 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 643/1274] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:38:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:38:51.355331 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b19014-8b03-48cf-986f-84172d24ede7 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:51.355737 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b19014-8b03-48cf-986f-84172d24ede7 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:51.360133 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b19014-8b03-48cf-986f-84172d24ede7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:51.372034 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-f0b19014-8b03-48cf-986f-84172d24ede7 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:38:51.373199 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-f0b19014-8b03-48cf-986f-84172d24ede7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:51.437381 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 632/1275] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:38:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:38:51.461389 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ff2d6-b5d4-4b8a-a7af-0aace85503c1 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:51.563103 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ff2d6-b5d4-4b8a-a7af-0aace85503c1 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:51.581134 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:51.599019 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07846957-a5fa-450f-982d-f75aa6b67f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:51.599744 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07846957-a5fa-450f-982d-f75aa6b67f28 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:51.600025 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07846957-a5fa-450f-982d-f75aa6b67f28 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:51.600300 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07846957-a5fa-450f-982d-f75aa6b67f28 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:51.610346 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07846957-a5fa-450f-982d-f75aa6b67f28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 22:38:51.618474 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-07846957-a5fa-450f-982d-f75aa6b67f28 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:38:51.618474 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:38:51.618474 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:38:51.618474 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:38:51.618474 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:38:51.618474 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:38:51.618474 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:38:51.618474 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:38:51.618474 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:38:51.618474 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:38:51.618474 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:38:51.618474 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:38:51.618474 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 22:38:51.618474 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 22:38:51.618474 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 22:38:51.618474 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 22:38:51.619620 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 22:38:51.619620 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 22:38:51.619620 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 22:38:51.619620 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 22:38:51.619620 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:38:51.619620 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:38:51.619620 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:38:51.619620 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:38:51.619620 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:38:51.619620 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:38:51.619620 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:38:51.619620 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:38:51.619620 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:38:51.619620 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:38:51.619620 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:38:51.619620 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:38:51.620836 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:38:51.620836 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:38:51.620836 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:38:51.620836 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:38:51.620836 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 22:38:51.620836 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 22:38:51.620836 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 22:38:51.620836 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 22:38:51.620836 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 22:38:51.620836 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 22:38:51.620836 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 22:38:51.620836 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:38:51.620836 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 644/1276] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Jul 21 22:38:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:38:51.625807 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b19014-8b03-48cf-986f-84172d24ede7 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:51.705716 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:51.724712 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b19014-8b03-48cf-986f-84172d24ede7 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:51.758477 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa09db-724d-4475-a970-a6df3e2a97d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:51.759166 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa09db-724d-4475-a970-a6df3e2a97d4 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:51.759448 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa09db-724d-4475-a970-a6df3e2a97d4 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:51.759725 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa09db-724d-4475-a970-a6df3e2a97d4 None admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:51.760473 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aa09db-724d-4475-a970-a6df3e2a97d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:38:51.760829 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aa09db-724d-4475-a970-a6df3e2a97d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:51.762180 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aa09db-724d-4475-a970-a6df3e2a97d4 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:51.770251 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 633/1277] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Jul 21 22:38:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:38:51.776156 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07846957-a5fa-450f-982d-f75aa6b67f28 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:51.877488 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07846957-a5fa-450f-982d-f75aa6b67f28 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:51.910353 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8ca507-c10a-48f7-8cbc-e17c9630b9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:51.911165 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8ca507-c10a-48f7-8cbc-e17c9630b9cf None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:51.911450 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8ca507-c10a-48f7-8cbc-e17c9630b9cf None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:51.911726 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8ca507-c10a-48f7-8cbc-e17c9630b9cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:51.922020 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8ca507-c10a-48f7-8cbc-e17c9630b9cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:51.924256 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8ca507-c10a-48f7-8cbc-e17c9630b9cf None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:51.941655 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-7b8ca507-c10a-48f7-8cbc-e17c9630b9cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:38:51.941655 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:38:51.941655 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:38:51.941655 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:38:51.941655 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:38:51.941655 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:38:51.941655 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:38:51.941655 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:38:51.941655 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:38:51.941655 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:38:51.941655 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:38:51.941655 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:38:51.941655 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:38:51.941655 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:38:51.941655 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:38:51.942765 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:38:51.942765 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:38:51.942765 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:38:51.942765 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:38:51.942765 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:38:51.942765 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:38:51.942765 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:38:51.942765 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:38:51.942765 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:38:51.942765 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:38:51.942765 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:38:51.942765 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:38:51.942765 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:38:51.942765 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:38:51.942765 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:38:51.942765 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:38:51.942765 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:38:51.944212 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:38:51.944212 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:38:51.944212 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:38:51.944212 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:38:51.944212 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 645/1278] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:38:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:38:51.948647 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa09db-724d-4475-a970-a6df3e2a97d4 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:52.049268 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa09db-724d-4475-a970-a6df3e2a97d4 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:52.084169 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48136177-8b04-454a-b217-3b339e166c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:52.084854 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48136177-8b04-454a-b217-3b339e166c3f None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:52.085141 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48136177-8b04-454a-b217-3b339e166c3f None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:52.085419 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48136177-8b04-454a-b217-3b339e166c3f None admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:52.086219 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48136177-8b04-454a-b217-3b339e166c3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:38:52.086523 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48136177-8b04-454a-b217-3b339e166c3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:52.087913 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48136177-8b04-454a-b217-3b339e166c3f None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:52.100319 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 634/1279] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:38:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:38:52.106079 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8ca507-c10a-48f7-8cbc-e17c9630b9cf None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:52.137065 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:52.207599 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8ca507-c10a-48f7-8cbc-e17c9630b9cf None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:52.242726 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c607579d-4320-4559-b1a1-8e674a64eb10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:52.243556 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c607579d-4320-4559-b1a1-8e674a64eb10 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:52.243845 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c607579d-4320-4559-b1a1-8e674a64eb10 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:52.244123 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c607579d-4320-4559-b1a1-8e674a64eb10 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:52.251600 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c607579d-4320-4559-b1a1-8e674a64eb10 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:52.253027 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c607579d-4320-4559-b1a1-8e674a64eb10 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:52.267089 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-c607579d-4320-4559-b1a1-8e674a64eb10 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 22:38:52.267089 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:38:52.267089 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:38:52.267089 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:38:52.267089 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:38:52.267089 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:38:52.267089 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:38:52.267089 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:38:52.267089 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:38:52.267089 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:38:52.267089 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:38:52.267089 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:38:52.267089 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 22:38:52.267089 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 22:38:52.267089 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 22:38:52.267089 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 22:38:52.268747 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:38:52.268747 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:38:52.268747 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 22:38:52.268747 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:38:52.268747 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 22:38:52.268747 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 22:38:52.268747 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:38:52.268747 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:38:52.268747 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:38:52.268747 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:38:52.268747 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:38:52.268747 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:38:52.268747 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:38:52.268747 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:38:52.268747 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:38:52.268747 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:38:52.268747 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:38:52.268747 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:38:52.269926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:38:52.269926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:38:52.269926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 22:38:52.269926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 22:38:52.269926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 22:38:52.269926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 22:38:52.269926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 22:38:52.269926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 22:38:52.269926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 22:38:52.269926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:38:52.269926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 646/1280] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:38:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:38:52.274677 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48136177-8b04-454a-b217-3b339e166c3f None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:52.278462 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:52.375063 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48136177-8b04-454a-b217-3b339e166c3f None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:52.408679 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9675e1f4-d852-45a2-baf4-4b2168bb24d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:52.409277 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9675e1f4-d852-45a2-baf4-4b2168bb24d7 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:52.409559 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9675e1f4-d852-45a2-baf4-4b2168bb24d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:52.409835 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9675e1f4-d852-45a2-baf4-4b2168bb24d7 None admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:52.410668 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9675e1f4-d852-45a2-baf4-4b2168bb24d7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:38:52.411009 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9675e1f4-d852-45a2-baf4-4b2168bb24d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:52.412454 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9675e1f4-d852-45a2-baf4-4b2168bb24d7 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:52.429377 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 635/1281] 10.222.0.52 () {56 vars in 1143 bytes} [Thu Jul 21 22:38:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:38:52.435282 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c607579d-4320-4559-b1a1-8e674a64eb10 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:52.530213 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:52.540246 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c607579d-4320-4559-b1a1-8e674a64eb10 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:52.574233 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22716116-62e7-4b0d-9b0f-918b891d6c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:52.574924 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22716116-62e7-4b0d-9b0f-918b891d6c54 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:52.575211 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22716116-62e7-4b0d-9b0f-918b891d6c54 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:52.575488 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22716116-62e7-4b0d-9b0f-918b891d6c54 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:52.642953 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22716116-62e7-4b0d-9b0f-918b891d6c54 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:52.644435 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22716116-62e7-4b0d-9b0f-918b891d6c54 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:52.731540 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:52.742112 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 647/1282] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Jul 21 22:38:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 22:38:52.748307 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9675e1f4-d852-45a2-baf4-4b2168bb24d7 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:52.810493 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:52.848238 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9675e1f4-d852-45a2-baf4-4b2168bb24d7 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:52.882531 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eb6a6f-c60c-401b-bbe8-80cc36fe095f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:52.883338 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb6a6f-c60c-401b-bbe8-80cc36fe095f None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:52.883621 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb6a6f-c60c-401b-bbe8-80cc36fe095f None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:52.883895 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb6a6f-c60c-401b-bbe8-80cc36fe095f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:52.894437 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eb6a6f-c60c-401b-bbe8-80cc36fe095f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:52.895879 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eb6a6f-c60c-401b-bbe8-80cc36fe095f None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:52.912864 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-b0eb6a6f-c60c-401b-bbe8-80cc36fe095f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:38:52.912864 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:38:52.912864 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:38:52.912864 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:38:52.912864 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:38:52.912864 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:38:52.912864 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:38:52.912864 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:38:52.912864 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:38:52.912864 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:38:52.912864 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:38:52.912864 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:38:52.912864 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 22:38:52.912864 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 22:38:52.912864 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 22:38:52.913923 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 22:38:52.913923 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:38:52.913923 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:38:52.913923 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:38:52.913923 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:38:52.913923 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:38:52.913923 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:38:52.913923 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:38:52.913923 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:38:52.913923 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:38:52.913923 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:38:52.913923 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:38:52.913923 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:38:52.913923 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:38:52.913923 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:38:52.913923 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:38:52.913923 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:38:52.915095 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 22:38:52.915095 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 22:38:52.915095 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 22:38:52.915095 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:38:52.915095 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 636/1283] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:38:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:38:52.919906 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22716116-62e7-4b0d-9b0f-918b891d6c54 None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:53.018694 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22716116-62e7-4b0d-9b0f-918b891d6c54 None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:53.052453 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014def39-eeca-4adf-8190-b7cc6c8f5e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:53.053107 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014def39-eeca-4adf-8190-b7cc6c8f5e4a None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:53.053391 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014def39-eeca-4adf-8190-b7cc6c8f5e4a None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:53.053666 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014def39-eeca-4adf-8190-b7cc6c8f5e4a None admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:53.054373 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014def39-eeca-4adf-8190-b7cc6c8f5e4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:38:53.054676 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014def39-eeca-4adf-8190-b7cc6c8f5e4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:53.056065 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014def39-eeca-4adf-8190-b7cc6c8f5e4a None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:53.069215 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 648/1284] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Jul 21 22:38:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:38:53.075017 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eb6a6f-c60c-401b-bbe8-80cc36fe095f None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:53.182421 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eb6a6f-c60c-401b-bbe8-80cc36fe095f None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:53.198606 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:53.220886 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61366b55-62cd-4a79-8086-069fe695d513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:53.221711 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61366b55-62cd-4a79-8086-069fe695d513 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:53.221997 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61366b55-62cd-4a79-8086-069fe695d513 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:53.222278 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61366b55-62cd-4a79-8086-069fe695d513 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:53.230553 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61366b55-62cd-4a79-8086-069fe695d513 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:53.232309 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61366b55-62cd-4a79-8086-069fe695d513 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:53.248382 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application [None req-61366b55-62cd-4a79-8086-069fe695d513 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:38:53.248382 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:38:53.248382 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:38:53.248382 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:38:53.248382 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:38:53.248382 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:38:53.248382 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:38:53.248382 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:38:53.248382 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:38:53.248382 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:38:53.248382 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:38:53.248382 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:38:53.248382 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 22:38:53.248382 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 22:38:53.248382 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 22:38:53.249468 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 22:38:53.249468 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:38:53.249468 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:38:53.249468 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:38:53.249468 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:38:53.249468 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:38:53.249468 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:38:53.249468 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:38:53.249468 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application with Lock( Jul 21 22:38:53.249468 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:38:53.249468 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:38:53.249468 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:38:53.249468 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:38:53.249468 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:38:53.249468 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:38:53.249468 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:38:53.249468 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:38:53.250360 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 22:38:53.250360 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 22:38:53.250360 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 22:38:53.250360 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 22:38:53.250360 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 22:38:53.250360 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 22:38:53.250360 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 22:38:53.250360 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: ERROR keystone.server.flask.application Jul 21 22:38:53.250360 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 637/1285] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Jul 21 22:38:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:38:53.256404 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014def39-eeca-4adf-8190-b7cc6c8f5e4a None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:53.316984 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:53.368578 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014def39-eeca-4adf-8190-b7cc6c8f5e4a None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:53.407294 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25a2a62-edf1-4aca-8e06-897763e33f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:53.408552 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a2a62-edf1-4aca-8e06-897763e33f48 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:53.408860 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a2a62-edf1-4aca-8e06-897763e33f48 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:53.409141 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a2a62-edf1-4aca-8e06-897763e33f48 None admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:53.409960 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25a2a62-edf1-4aca-8e06-897763e33f48 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:38:53.410274 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25a2a62-edf1-4aca-8e06-897763e33f48 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:53.411789 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25a2a62-edf1-4aca-8e06-897763e33f48 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:53.425958 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 649/1286] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Jul 21 22:38:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:38:53.432626 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61366b55-62cd-4a79-8086-069fe695d513 None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:53.547483 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61366b55-62cd-4a79-8086-069fe695d513 None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:53.582369 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47523c8b-9e21-43ef-b51c-86e65bb0a9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:53.582991 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47523c8b-9e21-43ef-b51c-86e65bb0a9b4 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:53.583278 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47523c8b-9e21-43ef-b51c-86e65bb0a9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:53.587915 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47523c8b-9e21-43ef-b51c-86e65bb0a9b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:53.587915 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47523c8b-9e21-43ef-b51c-86e65bb0a9b4 None admin] RBAC: Adding query filter params (role.id=d1ddee590a964ff1bbb4ee8716173021, scope.project.id=aa11ab8229b642ce962193b74001d3bc, user.id=10c0bed169344a77bfad29624c5e5a8e) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:38:53.587915 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47523c8b-9e21-43ef-b51c-86e65bb0a9b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:53.587915 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47523c8b-9e21-43ef-b51c-86e65bb0a9b4 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:53.592655 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 638/1287] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Jul 21 22:38:53 2022] GET /identity/v3/role_assignments?user.id=10c0bed169344a77bfad29624c5e5a8e&scope.project.id=aa11ab8229b642ce962193b74001d3bc&role.id=d1ddee590a964ff1bbb4ee8716173021 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:38:55.680189 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379dc59b-33a6-40a6-87dd-1d196340c2c3 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:55.680708 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379dc59b-33a6-40a6-87dd-1d196340c2c3 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:55.680800 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379dc59b-33a6-40a6-87dd-1d196340c2c3 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:55.682102 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 650/1288] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:38:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:38:55.687499 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4462235-ece4-42f7-b359-0159762cfa88 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:55.687916 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4462235-ece4-42f7-b359-0159762cfa88 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:55.688055 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4462235-ece4-42f7-b359-0159762cfa88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:55.704334 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-c4462235-ece4-42f7-b359-0159762cfa88 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:38:55.705510 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-c4462235-ece4-42f7-b359-0159762cfa88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:55.735614 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:55.772287 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 639/1289] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:38:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:38:55.795512 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379dc59b-33a6-40a6-87dd-1d196340c2c3 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:55.895630 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379dc59b-33a6-40a6-87dd-1d196340c2c3 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:55.906053 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:55.929419 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc8b93-6cba-4731-a58d-b358ea4336ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:55.930218 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc8b93-6cba-4731-a58d-b358ea4336ac None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:55.930502 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc8b93-6cba-4731-a58d-b358ea4336ac None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:55.930793 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc8b93-6cba-4731-a58d-b358ea4336ac None admin] PATH_INFO: `/v3/services/placement` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:55.931457 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddc8b93-6cba-4731-a58d-b358ea4336ac None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:55.932815 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddc8b93-6cba-4731-a58d-b358ea4336ac None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:55.944523 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-5ddc8b93-6cba-4731-a58d-b358ea4336ac None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 22:38:55.944523 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:38:55.944523 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:38:55.944523 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:38:55.944523 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:38:55.944523 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:38:55.944523 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:38:55.944523 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:38:55.944523 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 22:38:55.944523 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:38:55.944523 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:38:55.944523 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:38:55.944523 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:38:55.944523 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:38:55.944523 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:38:55.944523 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:38:55.944523 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:38:55.945742 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:38:55.945742 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:38:55.945742 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:38:55.945742 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:38:55.945742 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:38:55.945742 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:38:55.945742 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:38:55.945742 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:38:55.945742 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:38:55.945742 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:38:55.945742 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:38:55.945742 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:38:55.945742 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:38:55.945742 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:38:55.945742 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:38:55.945742 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:38:55.946736 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:38:55.946736 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:38:55.946736 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:38:55.946736 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:38:55.946736 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:38:55.946736 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:38:55.946736 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:38:55.946736 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:38:55.946736 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:38:55.946736 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:38:55.946736 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:38:55.946736 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:38:55.946736 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 22:38:55.946736 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:38:55.946736 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 651/1290] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Jul 21 22:38:55 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:38:55.951836 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4462235-ece4-42f7-b359-0159762cfa88 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:56.050462 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4462235-ece4-42f7-b359-0159762cfa88 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:56.083596 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f083a7-72b3-4105-a07d-0028a9c02d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:56.085268 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f083a7-72b3-4105-a07d-0028a9c02d03 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:56.085268 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f083a7-72b3-4105-a07d-0028a9c02d03 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:56.085268 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f083a7-72b3-4105-a07d-0028a9c02d03 None admin] PATH_INFO: `/v3/services` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:56.085581 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f083a7-72b3-4105-a07d-0028a9c02d03 None admin] RBAC: Adding query filter params (name=placement) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:38:56.085766 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f083a7-72b3-4105-a07d-0028a9c02d03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:56.087168 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f083a7-72b3-4105-a07d-0028a9c02d03 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:56.091967 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 640/1291] 10.222.0.52 () {56 vars in 1116 bytes} [Thu Jul 21 22:38:55 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:38:56.097696 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc8b93-6cba-4731-a58d-b358ea4336ac None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:56.197387 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc8b93-6cba-4731-a58d-b358ea4336ac None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:56.229818 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d810919-71ce-47aa-b0c5-f0971b28a0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:56.230464 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d810919-71ce-47aa-b0c5-f0971b28a0a6 None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:56.230744 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d810919-71ce-47aa-b0c5-f0971b28a0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:56.231045 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d810919-71ce-47aa-b0c5-f0971b28a0a6 None admin] PATH_INFO: `/v3/services` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:56.231735 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d810919-71ce-47aa-b0c5-f0971b28a0a6 None admin] RBAC: Adding query filter params (type=placement) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:38:56.232036 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d810919-71ce-47aa-b0c5-f0971b28a0a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:56.233374 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d810919-71ce-47aa-b0c5-f0971b28a0a6 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:56.238284 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 652/1292] 10.222.0.52 () {56 vars in 1116 bytes} [Thu Jul 21 22:38:56 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:38:58.302435 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a5edf-f699-4231-aa81-4d403c2f772a None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:58.303006 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a5edf-f699-4231-aa81-4d403c2f772a None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:58.303006 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a5edf-f699-4231-aa81-4d403c2f772a None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:58.304392 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 641/1293] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:38:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:38:58.309721 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952f000-5254-4d0a-b132-40948be32255 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:58.310094 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952f000-5254-4d0a-b132-40948be32255 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:58.310270 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952f000-5254-4d0a-b132-40948be32255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:58.326653 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-6952f000-5254-4d0a-b132-40948be32255 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:38:58.327847 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-6952f000-5254-4d0a-b132-40948be32255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:38:58.358367 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:58.395199 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 653/1294] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:38:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:38:58.418874 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4a5edf-f699-4231-aa81-4d403c2f772a None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:38:58.449418 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:38:58.524707 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4a5edf-f699-4231-aa81-4d403c2f772a None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:38:58.559116 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037a054c-c82d-476b-aca6-9fea94f554bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:38:58.559703 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a054c-c82d-476b-aca6-9fea94f554bc None admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:38:58.560043 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a054c-c82d-476b-aca6-9fea94f554bc None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:38:58.560312 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a054c-c82d-476b-aca6-9fea94f554bc None admin] PATH_INFO: `/v3/services` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:38:58.561021 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037a054c-c82d-476b-aca6-9fea94f554bc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:38:58.562403 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037a054c-c82d-476b-aca6-9fea94f554bc None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:38:58.642368 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 642/1295] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Jul 21 22:38:58 2022] POST /identity/v3/services => generated 246 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:39:00.701685 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf8f6b-3e35-4807-9734-3377e8754230 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:00.702232 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf8f6b-3e35-4807-9734-3377e8754230 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:00.702232 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf8f6b-3e35-4807-9734-3377e8754230 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:00.703652 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 654/1296] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:39:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:39:00.708970 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec22205d-cd0e-4cd3-b5b9-166fe0c35c71 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:00.709384 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec22205d-cd0e-4cd3-b5b9-166fe0c35c71 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:00.709534 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec22205d-cd0e-4cd3-b5b9-166fe0c35c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:00.726142 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-ec22205d-cd0e-4cd3-b5b9-166fe0c35c71 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:39:00.727354 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-ec22205d-cd0e-4cd3-b5b9-166fe0c35c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:00.799717 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 643/1297] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:39:00 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:39:00.822910 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bf8f6b-3e35-4807-9734-3377e8754230 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:00.924385 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bf8f6b-3e35-4807-9734-3377e8754230 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:00.957213 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d0a04f-a753-489f-a5df-43539eed520b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:00.958015 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0a04f-a753-489f-a5df-43539eed520b None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:00.958302 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0a04f-a753-489f-a5df-43539eed520b None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:00.958580 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0a04f-a753-489f-a5df-43539eed520b None admin] PATH_INFO: `/v3/services/placement` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:00.959261 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d0a04f-a753-489f-a5df-43539eed520b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:00.960620 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d0a04f-a753-489f-a5df-43539eed520b None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:00.972991 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-68d0a04f-a753-489f-a5df-43539eed520b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 22:39:00.972991 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:39:00.972991 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:39:00.972991 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:39:00.972991 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:39:00.972991 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:39:00.972991 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:39:00.972991 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:39:00.972991 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 22:39:00.972991 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:39:00.972991 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:39:00.972991 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:39:00.972991 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:39:00.972991 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:39:00.972991 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:39:00.972991 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:39:00.972991 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:39:00.974431 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:39:00.974431 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:39:00.974431 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:39:00.974431 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:39:00.974431 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:39:00.974431 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:39:00.974431 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:39:00.974431 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:39:00.974431 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:39:00.974431 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:39:00.974431 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:39:00.974431 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:39:00.974431 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:39:00.974431 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:39:00.974431 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:39:00.974431 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:39:00.975515 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:39:00.975515 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:39:00.975515 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:39:00.975515 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:39:00.975515 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:39:00.975515 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:39:00.975515 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:39:00.975515 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:39:00.975515 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:39:00.975515 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:39:00.975515 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:39:00.975515 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:39:00.975515 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 22:39:00.975515 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:39:00.975515 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 655/1298] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Jul 21 22:39:00 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:39:00.980492 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec22205d-cd0e-4cd3-b5b9-166fe0c35c71 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:00.999657 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:39:01.078274 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec22205d-cd0e-4cd3-b5b9-166fe0c35c71 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:01.113074 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102a11b7-97fb-46f9-974b-4a2f48846e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:01.113726 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a11b7-97fb-46f9-974b-4a2f48846e0e None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:01.114004 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a11b7-97fb-46f9-974b-4a2f48846e0e None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:01.114278 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a11b7-97fb-46f9-974b-4a2f48846e0e None admin] PATH_INFO: `/v3/services` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:01.115001 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102a11b7-97fb-46f9-974b-4a2f48846e0e None admin] RBAC: Adding query filter params (name=placement) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:39:01.115304 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102a11b7-97fb-46f9-974b-4a2f48846e0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:01.116648 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102a11b7-97fb-46f9-974b-4a2f48846e0e None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:01.121918 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 644/1299] 10.222.0.52 () {56 vars in 1116 bytes} [Thu Jul 21 22:39:00 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:39:01.128161 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d0a04f-a753-489f-a5df-43539eed520b None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:01.159689 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:39:01.228974 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d0a04f-a753-489f-a5df-43539eed520b None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:01.261679 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b40f16-1c8b-4491-8553-0b198cb52e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:01.262325 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b40f16-1c8b-4491-8553-0b198cb52e2f None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:01.262604 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b40f16-1c8b-4491-8553-0b198cb52e2f None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:01.262903 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b40f16-1c8b-4491-8553-0b198cb52e2f None admin] PATH_INFO: `/v3/endpoints` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:01.263736 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b40f16-1c8b-4491-8553-0b198cb52e2f None admin] RBAC: Adding query filter params (interface=public, service_id=d2c350b869fe4136973d24c977642f1e, region_id=RegionOne) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:39:01.264046 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b40f16-1c8b-4491-8553-0b198cb52e2f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:01.265401 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b40f16-1c8b-4491-8553-0b198cb52e2f None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:01.270959 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 656/1300] 10.222.0.52 () {56 vars in 1251 bytes} [Thu Jul 21 22:39:01 2022] GET /identity/v3/endpoints?service_id=d2c350b869fe4136973d24c977642f1e&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:39:01.276787 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102a11b7-97fb-46f9-974b-4a2f48846e0e None admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:01.358863 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:39:01.376253 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102a11b7-97fb-46f9-974b-4a2f48846e0e None admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:01.409663 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab30b3a-e935-4fa1-adcb-f9b4d3f70205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:01.410317 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab30b3a-e935-4fa1-adcb-f9b4d3f70205 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:01.410604 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab30b3a-e935-4fa1-adcb-f9b4d3f70205 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:01.410922 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab30b3a-e935-4fa1-adcb-f9b4d3f70205 None admin] PATH_INFO: `/v3/services` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:01.411602 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab30b3a-e935-4fa1-adcb-f9b4d3f70205 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:01.413004 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab30b3a-e935-4fa1-adcb-f9b4d3f70205 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:01.419714 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 645/1301] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Jul 21 22:39:01 2022] GET /identity/v3/services => generated 2191 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:39:03.464717 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19901750-c792-4c8d-aab4-ad5794358cd4 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:03.465294 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19901750-c792-4c8d-aab4-ad5794358cd4 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:03.465294 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19901750-c792-4c8d-aab4-ad5794358cd4 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:03.466628 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 657/1302] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:39:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:39:03.471770 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b36bbe-b355-473b-9b37-adb842ee2c7b None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:03.472160 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b36bbe-b355-473b-9b37-adb842ee2c7b None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:03.472313 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b36bbe-b355-473b-9b37-adb842ee2c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:03.489924 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-99b36bbe-b355-473b-9b37-adb842ee2c7b None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:39:03.491130 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-99b36bbe-b355-473b-9b37-adb842ee2c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:39:03.557421 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 646/1303] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:39:03 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 22:39:03.581358 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19901750-c792-4c8d-aab4-ad5794358cd4 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:03.675758 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:39:03.682631 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19901750-c792-4c8d-aab4-ad5794358cd4 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:03.716614 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bf48d5-4e2d-4de2-8c10-4de4eb0b2c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:03.717339 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf48d5-4e2d-4de2-8c10-4de4eb0b2c8e None admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:03.717624 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf48d5-4e2d-4de2-8c10-4de4eb0b2c8e None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:03.717902 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf48d5-4e2d-4de2-8c10-4de4eb0b2c8e None admin] PATH_INFO: `/v3/services/placement` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:03.718551 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bf48d5-4e2d-4de2-8c10-4de4eb0b2c8e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:03.719945 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bf48d5-4e2d-4de2-8c10-4de4eb0b2c8e None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:03.731873 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application [None req-a1bf48d5-4e2d-4de2-8c10-4de4eb0b2c8e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 22:39:03.731873 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:39:03.731873 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 22:39:03.731873 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 22:39:03.731873 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 22:39:03.731873 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 22:39:03.731873 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 22:39:03.731873 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:39:03.731873 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 22:39:03.731873 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:39:03.731873 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 22:39:03.731873 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:39:03.731873 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 22:39:03.731873 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 22:39:03.731873 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 22:39:03.731873 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 22:39:03.731873 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 22:39:03.732957 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 22:39:03.732957 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 22:39:03.732957 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 22:39:03.732957 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 22:39:03.732957 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 22:39:03.732957 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 22:39:03.732957 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 22:39:03.732957 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 22:39:03.732957 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 22:39:03.732957 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 22:39:03.732957 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 22:39:03.732957 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 22:39:03.732957 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 22:39:03.732957 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 22:39:03.732957 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 22:39:03.732957 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 22:39:03.733813 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 22:39:03.733813 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application with Lock( Jul 21 22:39:03.733813 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 22:39:03.733813 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self._enter() Jul 21 22:39:03.733813 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 22:39:03.733813 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 22:39:03.733813 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 22:39:03.733813 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application return self.creator() Jul 21 22:39:03.733813 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 22:39:03.733813 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application created_value = creator( Jul 21 22:39:03.733813 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 22:39:03.733813 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 22:39:03.733813 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 22:39:03.733813 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: ERROR keystone.server.flask.application Jul 21 22:39:03.733813 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 658/1304] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Jul 21 22:39:03 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 22:39:03.739763 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b36bbe-b355-473b-9b37-adb842ee2c7b None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:03.819286 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:39:03.841353 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b36bbe-b355-473b-9b37-adb842ee2c7b None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:03.874355 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12be98b1-a9f4-4912-8b9c-7524e30509e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:03.875030 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12be98b1-a9f4-4912-8b9c-7524e30509e3 None admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:03.875311 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12be98b1-a9f4-4912-8b9c-7524e30509e3 None admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:03.875580 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12be98b1-a9f4-4912-8b9c-7524e30509e3 None admin] PATH_INFO: `/v3/services` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:03.876273 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12be98b1-a9f4-4912-8b9c-7524e30509e3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:39:03.877199 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12be98b1-a9f4-4912-8b9c-7524e30509e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:03.878543 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12be98b1-a9f4-4912-8b9c-7524e30509e3 None admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:03.884417 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 647/1305] 10.222.0.52 () {56 vars in 1116 bytes} [Thu Jul 21 22:39:03 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:39:03.890351 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bf48d5-4e2d-4de2-8c10-4de4eb0b2c8e None admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:39:03.989516 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bf48d5-4e2d-4de2-8c10-4de4eb0b2c8e None admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:39:04.022685 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e202122a-48f9-4273-8145-06ef4722e298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:39:04.023359 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e202122a-48f9-4273-8145-06ef4722e298 None admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:39:04.023643 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e202122a-48f9-4273-8145-06ef4722e298 None admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:39:04.023922 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e202122a-48f9-4273-8145-06ef4722e298 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:39:04.024654 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e202122a-48f9-4273-8145-06ef4722e298 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:39:04.026002 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e202122a-48f9-4273-8145-06ef4722e298 None admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:39:04.099312 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 659/1306] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Jul 21 22:39:03 2022] POST /identity/v3/endpoints => generated 331 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:39:13.882886 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=30 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:39:14.094307 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:44:44.962331 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb0aae-f096-491e-b153-ab8cfa754cab None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:44:44.963398 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb0aae-f096-491e-b153-ab8cfa754cab None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:44:44.963925 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb0aae-f096-491e-b153-ab8cfa754cab None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:44:44.971420 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc744c5-861d-4a93-802c-acbdd528be88 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:44:44.972312 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc744c5-861d-4a93-802c-acbdd528be88 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:44:44.972823 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc744c5-861d-4a93-802c-acbdd528be88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:44:44.976759 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 648/1307] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:44:44 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:44:45.059809 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-cbc744c5-861d-4a93-802c-acbdd528be88 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:44:45.061380 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-cbc744c5-861d-4a93-802c-acbdd528be88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:44:45.218323 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 660/1308] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:44:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:44:45.260281 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada86cbb-fd64-4f86-9dfc-bd5c1a912abd None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:44:45.260851 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada86cbb-fd64-4f86-9dfc-bd5c1a912abd None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:44:45.261396 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada86cbb-fd64-4f86-9dfc-bd5c1a912abd None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:44:45.269222 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 649/1309] 10.222.0.52 () {54 vars in 965 bytes} [Thu Jul 21 22:44:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:44:45.270436 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45e273-5f14-42af-a3b5-7a4802d3e1d9 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:44:45.270630 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45e273-5f14-42af-a3b5-7a4802d3e1d9 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:44:45.270929 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45e273-5f14-42af-a3b5-7a4802d3e1d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:44:45.387700 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-7f45e273-5f14-42af-a3b5-7a4802d3e1d9 None None] MFA Rules not processed for user `48c3e33f24ff4d66a72f06848d769ac7`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:44:45.389600 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-7f45e273-5f14-42af-a3b5-7a4802d3e1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:44:45.423094 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:44:45.492457 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 661/1310] 10.222.0.52 () {58 vars in 1062 bytes} [Thu Jul 21 22:44:45 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:44:45.501647 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada86cbb-fd64-4f86-9dfc-bd5c1a912abd None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:44:45.503624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-ada86cbb-fd64-4f86-9dfc-bd5c1a912abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:44:45.625840 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada86cbb-fd64-4f86-9dfc-bd5c1a912abd None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:44:45.706117 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4798cf4d-1e96-4608-834d-fce3d022d63c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:44:45.706862 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4798cf4d-1e96-4608-834d-fce3d022d63c service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:44:45.707088 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4798cf4d-1e96-4608-834d-fce3d022d63c service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:44:45.707362 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4798cf4d-1e96-4608-834d-fce3d022d63c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:44:45.709044 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-4798cf4d-1e96-4608-834d-fce3d022d63c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:44:45.767331 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:44:45.822693 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4798cf4d-1e96-4608-834d-fce3d022d63c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:44:45.824254 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4798cf4d-1e96-4608-834d-fce3d022d63c service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:44:45.845279 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 650/1311] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:44:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:44:48.456865 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b7d26-8e30-4cb6-bf44-b80e32ce6f81 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:44:48.457476 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b7d26-8e30-4cb6-bf44-b80e32ce6f81 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:44:48.457476 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b7d26-8e30-4cb6-bf44-b80e32ce6f81 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:44:48.459335 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 662/1312] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:44:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:44:48.465278 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b28033-5739-4757-9956-4a2dd0aa7039 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:44:48.465920 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b28033-5739-4757-9956-4a2dd0aa7039 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:44:48.466065 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b28033-5739-4757-9956-4a2dd0aa7039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:44:48.549455 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-f2b28033-5739-4757-9956-4a2dd0aa7039 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:44:48.550841 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-f2b28033-5739-4757-9956-4a2dd0aa7039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:44:48.655043 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 651/1313] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:44:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:44:48.693722 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3b7d26-8e30-4cb6-bf44-b80e32ce6f81 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:44:48.695304 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-de3b7d26-8e30-4cb6-bf44-b80e32ce6f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:44:48.807107 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3b7d26-8e30-4cb6-bf44-b80e32ce6f81 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:44:48.891493 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:44:48.893927 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d401822d-56f7-46fe-b687-97ee25faa61a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:44:48.894605 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d401822d-56f7-46fe-b687-97ee25faa61a service neutron] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:44:48.895177 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d401822d-56f7-46fe-b687-97ee25faa61a service neutron] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:44:48.895470 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d401822d-56f7-46fe-b687-97ee25faa61a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:44:48.897111 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-d401822d-56f7-46fe-b687-97ee25faa61a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:44:49.008501 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d401822d-56f7-46fe-b687-97ee25faa61a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:44:49.010023 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d401822d-56f7-46fe-b687-97ee25faa61a service neutron] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:44:49.029103 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 663/1314] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:44:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:44:51.528428 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e2981d-e9b1-4eec-82bd-d34b315ed5a3 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:44:51.528890 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e2981d-e9b1-4eec-82bd-d34b315ed5a3 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:44:51.528997 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e2981d-e9b1-4eec-82bd-d34b315ed5a3 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:44:51.530407 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 652/1315] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:44:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:44:51.536038 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f04eda-bb1b-4384-80f8-43cdbaa8a7e8 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:44:51.536414 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f04eda-bb1b-4384-80f8-43cdbaa8a7e8 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:44:51.536588 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f04eda-bb1b-4384-80f8-43cdbaa8a7e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:44:51.601173 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-f0f04eda-bb1b-4384-80f8-43cdbaa8a7e8 None None] MFA Rules not processed for user `8e9d5ca05dea4d9c9dad9a3a31239f82`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:44:51.602372 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-f0f04eda-bb1b-4384-80f8-43cdbaa8a7e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:44:51.652942 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:44:51.696564 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 664/1316] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:44:51 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:44:51.736309 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e2981d-e9b1-4eec-82bd-d34b315ed5a3 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:44:51.737843 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-c1e2981d-e9b1-4eec-82bd-d34b315ed5a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:44:51.747715 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:44:51.840669 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e2981d-e9b1-4eec-82bd-d34b315ed5a3 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:44:51.913135 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb406522-6fe9-432f-9487-62567314bfd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:44:51.913777 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb406522-6fe9-432f-9487-62567314bfd4 service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:44:51.914060 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb406522-6fe9-432f-9487-62567314bfd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:44:51.914334 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb406522-6fe9-432f-9487-62567314bfd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:44:51.916005 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-fb406522-6fe9-432f-9487-62567314bfd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:44:51.978099 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:44:52.023875 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb406522-6fe9-432f-9487-62567314bfd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:44:52.025234 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb406522-6fe9-432f-9487-62567314bfd4 service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:44:52.043873 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 653/1317] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:44:51 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:44:54.820901 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d11621-e776-45b4-af3b-01086c54b8ef None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:44:54.821408 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d11621-e776-45b4-af3b-01086c54b8ef None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:44:54.821408 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d11621-e776-45b4-af3b-01086c54b8ef None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:44:54.822747 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 665/1318] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:44:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:44:54.828771 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fadc5a-cde7-4a4b-a448-d16f03308051 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:44:54.829179 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fadc5a-cde7-4a4b-a448-d16f03308051 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:44:54.829386 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fadc5a-cde7-4a4b-a448-d16f03308051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:44:54.893951 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-31fadc5a-cde7-4a4b-a448-d16f03308051 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:44:54.895181 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-31fadc5a-cde7-4a4b-a448-d16f03308051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:44:54.991327 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 654/1319] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:44:54 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:44:55.031299 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d11621-e776-45b4-af3b-01086c54b8ef None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:44:55.032842 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-89d11621-e776-45b4-af3b-01086c54b8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:44:55.134959 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d11621-e776-45b4-af3b-01086c54b8ef None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:44:55.208823 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e14192b-1db3-4efe-887e-d9db366c500b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:44:55.209462 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e14192b-1db3-4efe-887e-d9db366c500b service neutron] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:44:55.209740 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e14192b-1db3-4efe-887e-d9db366c500b service neutron] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:44:55.210010 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e14192b-1db3-4efe-887e-d9db366c500b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:44:55.211662 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-5e14192b-1db3-4efe-887e-d9db366c500b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:44:55.219573 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:44:55.319806 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e14192b-1db3-4efe-887e-d9db366c500b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:44:55.321268 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e14192b-1db3-4efe-887e-d9db366c500b service neutron] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:44:55.339030 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 666/1320] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:44:55 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:44:58.321505 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8232a56-5cc3-4b7a-a756-d91e71a0ebfe None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:44:58.322034 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8232a56-5cc3-4b7a-a756-d91e71a0ebfe None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:44:58.322150 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8232a56-5cc3-4b7a-a756-d91e71a0ebfe None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:44:58.323485 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 655/1321] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:44:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:44:58.329446 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505af7ac-8ad4-46f3-a89a-bcf55c8374e6 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:44:58.329834 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505af7ac-8ad4-46f3-a89a-bcf55c8374e6 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:44:58.330018 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505af7ac-8ad4-46f3-a89a-bcf55c8374e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:44:58.395004 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-505af7ac-8ad4-46f3-a89a-bcf55c8374e6 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:44:58.396219 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-505af7ac-8ad4-46f3-a89a-bcf55c8374e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:44:58.454047 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:44:58.497482 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 667/1322] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:44:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:44:58.533666 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8232a56-5cc3-4b7a-a756-d91e71a0ebfe None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:44:58.535263 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-a8232a56-5cc3-4b7a-a756-d91e71a0ebfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:44:58.555590 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:44:58.650453 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8232a56-5cc3-4b7a-a756-d91e71a0ebfe None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:44:58.728460 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447e337e-162a-477a-8364-f58498ae60a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:44:58.729119 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e337e-162a-477a-8364-f58498ae60a6 service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:44:58.729402 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e337e-162a-477a-8364-f58498ae60a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:44:58.729681 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e337e-162a-477a-8364-f58498ae60a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:44:58.731398 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-447e337e-162a-477a-8364-f58498ae60a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:44:58.806185 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:44:58.845589 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447e337e-162a-477a-8364-f58498ae60a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:44:58.847030 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447e337e-162a-477a-8364-f58498ae60a6 service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:44:58.865649 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 656/1323] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:44:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:45:00.969876 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6bcc22-e2ff-48f0-a635-03f3937df4e2 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:00.970436 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6bcc22-e2ff-48f0-a635-03f3937df4e2 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:00.970436 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6bcc22-e2ff-48f0-a635-03f3937df4e2 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:00.971808 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 668/1324] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:45:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:45:00.976930 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4498f64-3558-4900-bf76-32ed6d9277a2 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:00.977368 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4498f64-3558-4900-bf76-32ed6d9277a2 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:00.977624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4498f64-3558-4900-bf76-32ed6d9277a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:01.043764 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-a4498f64-3558-4900-bf76-32ed6d9277a2 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:45:01.044958 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-a4498f64-3558-4900-bf76-32ed6d9277a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:45:01.142120 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 657/1325] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:45:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:45:01.192393 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fdad2b-1d0f-4dc1-9d35-3723a8d14e15 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:01.192771 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fdad2b-1d0f-4dc1-9d35-3723a8d14e15 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:01.193139 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fdad2b-1d0f-4dc1-9d35-3723a8d14e15 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:01.194630 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 669/1326] 10.222.0.52 () {54 vars in 965 bytes} [Thu Jul 21 22:45:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:45:01.200592 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10f1f0-d356-4bbf-8c0b-35ac319e64b7 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:01.200882 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10f1f0-d356-4bbf-8c0b-35ac319e64b7 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:01.201162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10f1f0-d356-4bbf-8c0b-35ac319e64b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:01.235285 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:45:01.279479 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-ce10f1f0-d356-4bbf-8c0b-35ac319e64b7 None None] MFA Rules not processed for user `48c3e33f24ff4d66a72f06848d769ac7`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:45:01.280710 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-ce10f1f0-d356-4bbf-8c0b-35ac319e64b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:45:01.355062 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 658/1327] 10.222.0.52 () {58 vars in 1062 bytes} [Thu Jul 21 22:45:01 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:45:01.364331 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fdad2b-1d0f-4dc1-9d35-3723a8d14e15 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:45:01.365898 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-58fdad2b-1d0f-4dc1-9d35-3723a8d14e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:45:01.475717 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fdad2b-1d0f-4dc1-9d35-3723a8d14e15 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:45:01.556344 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab9848-195f-4068-b9de-869d59fed999 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:45:01.556991 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab9848-195f-4068-b9de-869d59fed999 service neutron] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:01.557270 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab9848-195f-4068-b9de-869d59fed999 service neutron] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:01.557544 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab9848-195f-4068-b9de-869d59fed999 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:01.559194 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-ecab9848-195f-4068-b9de-869d59fed999 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:45:01.576067 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:45:01.672377 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecab9848-195f-4068-b9de-869d59fed999 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:45:01.673806 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecab9848-195f-4068-b9de-869d59fed999 service neutron] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:45:01.692703 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 670/1328] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:45:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:45:04.011669 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a59fc-2a44-4282-bd12-44d8aa292d8b None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:04.012227 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a59fc-2a44-4282-bd12-44d8aa292d8b None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:04.012227 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a59fc-2a44-4282-bd12-44d8aa292d8b None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:04.013661 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 659/1329] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:45:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:45:04.019725 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643386a4-b708-4035-9400-8873ea8a6557 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:04.020108 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643386a4-b708-4035-9400-8873ea8a6557 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:04.020406 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643386a4-b708-4035-9400-8873ea8a6557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:04.087677 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-643386a4-b708-4035-9400-8873ea8a6557 None None] MFA Rules not processed for user `8e9d5ca05dea4d9c9dad9a3a31239f82`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:45:04.088993 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-643386a4-b708-4035-9400-8873ea8a6557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:45:04.157249 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:45:04.189412 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 671/1330] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:45:04 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:45:04.226571 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391a59fc-2a44-4282-bd12-44d8aa292d8b None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:45:04.228390 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-391a59fc-2a44-4282-bd12-44d8aa292d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:45:04.356850 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391a59fc-2a44-4282-bd12-44d8aa292d8b None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:45:04.379477 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:45:04.433169 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d5712e-8537-4ced-a185-bc30ad5329f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:45:04.433820 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d5712e-8537-4ced-a185-bc30ad5329f5 service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:04.434104 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d5712e-8537-4ced-a185-bc30ad5329f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:04.434381 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d5712e-8537-4ced-a185-bc30ad5329f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:04.436061 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-a5d5712e-8537-4ced-a185-bc30ad5329f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:45:04.551380 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d5712e-8537-4ced-a185-bc30ad5329f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:45:04.552743 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d5712e-8537-4ced-a185-bc30ad5329f5 service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:45:04.571351 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 660/1331] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:45:04 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:45:07.237849 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ff3a2-3e63-4860-aace-2f52ccbc2593 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:07.238354 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ff3a2-3e63-4860-aace-2f52ccbc2593 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:07.238354 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ff3a2-3e63-4860-aace-2f52ccbc2593 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:07.239694 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 672/1332] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:45:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:45:07.244886 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481be0b2-cc99-46a8-8bd9-d96dad3169e8 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:07.245307 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481be0b2-cc99-46a8-8bd9-d96dad3169e8 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:07.245449 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481be0b2-cc99-46a8-8bd9-d96dad3169e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:07.307109 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:45:07.310966 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-481be0b2-cc99-46a8-8bd9-d96dad3169e8 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:45:07.312176 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-481be0b2-cc99-46a8-8bd9-d96dad3169e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:45:07.409016 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 661/1333] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:45:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:45:07.445437 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5ff3a2-3e63-4860-aace-2f52ccbc2593 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:45:07.447005 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-0d5ff3a2-3e63-4860-aace-2f52ccbc2593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:45:07.551487 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5ff3a2-3e63-4860-aace-2f52ccbc2593 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:45:07.622957 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28b4f70-b037-46a3-aa04-a0430279ce4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:45:07.623631 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b4f70-b037-46a3-aa04-a0430279ce4a service neutron] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:07.623920 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b4f70-b037-46a3-aa04-a0430279ce4a service neutron] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:07.624191 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b4f70-b037-46a3-aa04-a0430279ce4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:07.625828 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-d28b4f70-b037-46a3-aa04-a0430279ce4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:45:07.651539 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:45:07.732812 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28b4f70-b037-46a3-aa04-a0430279ce4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:45:07.734273 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28b4f70-b037-46a3-aa04-a0430279ce4a service neutron] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:45:07.752372 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 673/1334] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:45:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:45:09.853611 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fc19f6-85c6-44b9-ace3-e1f8decc928e None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:09.854148 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fc19f6-85c6-44b9-ace3-e1f8decc928e None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:09.854148 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fc19f6-85c6-44b9-ace3-e1f8decc928e None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:09.855561 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 662/1335] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:45:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:45:09.861397 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2832c-3a85-4262-b3d5-ac60049fae5c None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:09.861796 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2832c-3a85-4262-b3d5-ac60049fae5c None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:09.861980 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2832c-3a85-4262-b3d5-ac60049fae5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:09.926482 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-4de2832c-3a85-4262-b3d5-ac60049fae5c None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:45:09.927718 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-4de2832c-3a85-4262-b3d5-ac60049fae5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:45:10.001670 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:45:10.023687 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 674/1336] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:45:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:45:10.070023 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fc19f6-85c6-44b9-ace3-e1f8decc928e None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:45:10.071613 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-13fc19f6-85c6-44b9-ace3-e1f8decc928e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:45:10.174137 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fc19f6-85c6-44b9-ace3-e1f8decc928e None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:45:10.201361 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:45:10.246639 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7264defc-2f7a-46e6-97b1-aa1406b48f86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:45:10.247305 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7264defc-2f7a-46e6-97b1-aa1406b48f86 service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:10.247588 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7264defc-2f7a-46e6-97b1-aa1406b48f86 service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:10.247864 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7264defc-2f7a-46e6-97b1-aa1406b48f86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:10.249597 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-7264defc-2f7a-46e6-97b1-aa1406b48f86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:45:10.355441 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7264defc-2f7a-46e6-97b1-aa1406b48f86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:45:10.356863 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7264defc-2f7a-46e6-97b1-aa1406b48f86 service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:45:10.374964 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 663/1337] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:45:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:45:13.056637 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e8db5-88fa-44b4-9003-e6896262b5df None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:13.057188 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e8db5-88fa-44b4-9003-e6896262b5df None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:13.057188 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e8db5-88fa-44b4-9003-e6896262b5df None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:13.058564 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 675/1338] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:45:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:45:13.063640 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b155976-1840-4c79-b443-dce5077230c5 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:13.064044 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b155976-1840-4c79-b443-dce5077230c5 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:13.064201 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b155976-1840-4c79-b443-dce5077230c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:13.129511 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-2b155976-1840-4c79-b443-dce5077230c5 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:45:13.130639 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-2b155976-1840-4c79-b443-dce5077230c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:45:13.139401 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:45:13.228492 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 664/1339] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:45:13 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:45:13.264653 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23e8db5-88fa-44b4-9003-e6896262b5df None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:45:13.266186 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-b23e8db5-88fa-44b4-9003-e6896262b5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:45:13.370481 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23e8db5-88fa-44b4-9003-e6896262b5df None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:45:13.444870 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0098f6-a348-46a6-a6b7-d214aa9ef1b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:45:13.445440 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0098f6-a348-46a6-a6b7-d214aa9ef1b5 service neutron] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:13.445726 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0098f6-a348-46a6-a6b7-d214aa9ef1b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:13.446001 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0098f6-a348-46a6-a6b7-d214aa9ef1b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:13.447684 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-ca0098f6-a348-46a6-a6b7-d214aa9ef1b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:45:13.484017 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:45:13.559007 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0098f6-a348-46a6-a6b7-d214aa9ef1b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:45:13.560485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0098f6-a348-46a6-a6b7-d214aa9ef1b5 service neutron] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:45:13.579001 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 676/1340] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:45:13 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:45:17.892348 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc35966-8f72-4599-8dd4-7a56557143d5 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:17.892842 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc35966-8f72-4599-8dd4-7a56557143d5 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:17.893161 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc35966-8f72-4599-8dd4-7a56557143d5 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:17.895098 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 665/1341] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:45:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:45:17.900636 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7a1fe-ca38-4db6-8770-171cf9fce09a None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:17.901075 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7a1fe-ca38-4db6-8770-171cf9fce09a None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:17.901246 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7a1fe-ca38-4db6-8770-171cf9fce09a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:17.969401 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-dbe7a1fe-ca38-4db6-8770-171cf9fce09a None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:45:17.970615 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-dbe7a1fe-ca38-4db6-8770-171cf9fce09a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:45:18.064258 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:45:18.075644 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 677/1342] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:45:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:45:18.115127 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc35966-8f72-4599-8dd4-7a56557143d5 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:45:18.116672 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-bfc35966-8f72-4599-8dd4-7a56557143d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:45:18.219666 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc35966-8f72-4599-8dd4-7a56557143d5 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:45:18.256556 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:45:18.292691 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a79ff43-c032-40c1-bdc6-51bbe0876a81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:45:18.293333 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a79ff43-c032-40c1-bdc6-51bbe0876a81 service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:18.293613 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a79ff43-c032-40c1-bdc6-51bbe0876a81 service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:18.293885 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a79ff43-c032-40c1-bdc6-51bbe0876a81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:18.296261 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-6a79ff43-c032-40c1-bdc6-51bbe0876a81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:45:18.402038 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a79ff43-c032-40c1-bdc6-51bbe0876a81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:45:18.403491 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a79ff43-c032-40c1-bdc6-51bbe0876a81 service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:45:18.422492 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 666/1343] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:45:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:45:21.349232 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8333a9-1da2-4874-995c-e61a931564eb None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:21.349762 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8333a9-1da2-4874-995c-e61a931564eb None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:21.349856 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8333a9-1da2-4874-995c-e61a931564eb None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:21.351213 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 678/1344] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:45:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:45:21.356743 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32744820-c51d-449e-8b9e-98b2e242a44d None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:21.357159 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32744820-c51d-449e-8b9e-98b2e242a44d None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:21.357819 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32744820-c51d-449e-8b9e-98b2e242a44d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:21.421790 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-32744820-c51d-449e-8b9e-98b2e242a44d None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:45:21.423034 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-32744820-c51d-449e-8b9e-98b2e242a44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:45:21.441700 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:45:21.521634 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 667/1345] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:45:21 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:45:21.559102 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8333a9-1da2-4874-995c-e61a931564eb None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:45:21.561232 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-fb8333a9-1da2-4874-995c-e61a931564eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:45:21.663943 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8333a9-1da2-4874-995c-e61a931564eb None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:45:21.741664 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca27b1-fabe-45fc-8b64-38d0383258c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:45:21.742320 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca27b1-fabe-45fc-8b64-38d0383258c5 service neutron] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:21.742607 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca27b1-fabe-45fc-8b64-38d0383258c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:21.742927 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca27b1-fabe-45fc-8b64-38d0383258c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:21.744584 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-54ca27b1-fabe-45fc-8b64-38d0383258c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:45:21.788452 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:45:21.856755 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ca27b1-fabe-45fc-8b64-38d0383258c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:45:21.858901 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ca27b1-fabe-45fc-8b64-38d0383258c5 service neutron] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:45:21.878539 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 679/1346] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:45:21 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:45:26.343963 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61e1dcf-a52c-4cac-856f-a0dfc16bc884 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:26.344498 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61e1dcf-a52c-4cac-856f-a0dfc16bc884 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:26.344498 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61e1dcf-a52c-4cac-856f-a0dfc16bc884 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:26.346030 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 668/1347] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:45:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:45:26.351224 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b40fb3-ee51-43aa-bec5-ff659916842b None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:26.351611 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b40fb3-ee51-43aa-bec5-ff659916842b None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:26.351773 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b40fb3-ee51-43aa-bec5-ff659916842b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:26.420558 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-65b40fb3-ee51-43aa-bec5-ff659916842b None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:45:26.421850 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-65b40fb3-ee51-43aa-bec5-ff659916842b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:45:26.526888 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:45:26.529920 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 680/1348] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:45:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:45:26.568726 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61e1dcf-a52c-4cac-856f-a0dfc16bc884 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:45:26.570260 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-f61e1dcf-a52c-4cac-856f-a0dfc16bc884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:45:26.683676 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61e1dcf-a52c-4cac-856f-a0dfc16bc884 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:45:26.735609 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:45:26.760713 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e925360c-40c4-4633-a243-cf4c6f04b258 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:45:26.761376 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925360c-40c4-4633-a243-cf4c6f04b258 service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:26.761653 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925360c-40c4-4633-a243-cf4c6f04b258 service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:26.761923 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925360c-40c4-4633-a243-cf4c6f04b258 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:26.764082 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-e925360c-40c4-4633-a243-cf4c6f04b258 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:45:26.871899 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e925360c-40c4-4633-a243-cf4c6f04b258 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:45:26.872485 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e925360c-40c4-4633-a243-cf4c6f04b258 service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:45:26.891839 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 669/1349] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:45:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:45:31.831051 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e6003-3372-4b18-8d10-b92654bac302 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:31.831551 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e6003-3372-4b18-8d10-b92654bac302 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:31.831551 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e6003-3372-4b18-8d10-b92654bac302 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:31.833044 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 681/1350] 10.222.0.52 () {54 vars in 921 bytes} [Thu Jul 21 22:45:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:45:31.837969 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba93bb5-a264-4a3f-b512-7506fd3180f2 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:31.838385 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba93bb5-a264-4a3f-b512-7506fd3180f2 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:31.838717 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba93bb5-a264-4a3f-b512-7506fd3180f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:31.979384 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-dba93bb5-a264-4a3f-b512-7506fd3180f2 None None] MFA Rules not processed for user `10c0bed169344a77bfad29624c5e5a8e`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:45:31.980925 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-dba93bb5-a264-4a3f-b512-7506fd3180f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:45:32.033181 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:45:32.106953 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 670/1351] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Jul 21 22:45:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:45:36.260117 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a817f22-aefe-4892-80ee-5e07515caa64 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:36.260681 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a817f22-aefe-4892-80ee-5e07515caa64 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:36.260681 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a817f22-aefe-4892-80ee-5e07515caa64 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:36.262214 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 682/1352] 10.222.0.52 () {54 vars in 921 bytes} [Thu Jul 21 22:45:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:45:36.267328 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395a562-0b48-4737-8209-5f7e439ffd51 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:36.268118 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395a562-0b48-4737-8209-5f7e439ffd51 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:36.268463 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395a562-0b48-4737-8209-5f7e439ffd51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:36.346590 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-f395a562-0b48-4737-8209-5f7e439ffd51 None None] MFA Rules not processed for user `10c0bed169344a77bfad29624c5e5a8e`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:45:36.348071 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-f395a562-0b48-4737-8209-5f7e439ffd51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:45:36.423839 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:45:36.461189 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 671/1353] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Jul 21 22:45:36 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:45:37.954900 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8beef57-237e-4ffd-92c3-8136a7da629a None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:37.955419 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8beef57-237e-4ffd-92c3-8136a7da629a None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:37.955579 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8beef57-237e-4ffd-92c3-8136a7da629a None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:37.956981 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 683/1354] 10.222.0.52 () {54 vars in 921 bytes} [Thu Jul 21 22:45:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:45:37.962093 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d0d2f5-bb60-47d2-bb5b-aa7ea7ed29a5 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:37.962524 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d0d2f5-bb60-47d2-bb5b-aa7ea7ed29a5 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:37.962677 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d0d2f5-bb60-47d2-bb5b-aa7ea7ed29a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:38.037180 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-e8d0d2f5-bb60-47d2-bb5b-aa7ea7ed29a5 None None] MFA Rules not processed for user `10c0bed169344a77bfad29624c5e5a8e`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:45:38.038374 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-e8d0d2f5-bb60-47d2-bb5b-aa7ea7ed29a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:45:38.145217 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 672/1355] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Jul 21 22:45:37 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:45:41.624545 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e075a09-5cea-4420-9dd3-b88439d7b341 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:41.625113 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e075a09-5cea-4420-9dd3-b88439d7b341 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:41.625113 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e075a09-5cea-4420-9dd3-b88439d7b341 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:41.626737 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 684/1356] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:45:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:45:41.632255 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee33bee-0ad4-4047-b653-41e5b298b5b1 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:41.632731 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee33bee-0ad4-4047-b653-41e5b298b5b1 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:41.632840 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee33bee-0ad4-4047-b653-41e5b298b5b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:41.671609 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:45:41.704618 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-2ee33bee-0ad4-4047-b653-41e5b298b5b1 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:45:41.705742 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-2ee33bee-0ad4-4047-b653-41e5b298b5b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:45:41.807133 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 673/1357] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:45:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:45:41.863153 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad698f5-9a4d-4572-b3ea-707d78d05696 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:41.863837 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad698f5-9a4d-4572-b3ea-707d78d05696 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:41.864319 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad698f5-9a4d-4572-b3ea-707d78d05696 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:41.866388 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 685/1358] 10.222.0.52 () {54 vars in 962 bytes} [Thu Jul 21 22:45:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:45:41.874107 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a143751e-5e5d-4753-9a17-df62558552a0 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:41.881931 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a143751e-5e5d-4753-9a17-df62558552a0 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:41.881931 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a143751e-5e5d-4753-9a17-df62558552a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:41.960497 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-a143751e-5e5d-4753-9a17-df62558552a0 None None] MFA Rules not processed for user `6bfdb655a2a44b33ab724daaf73a294b`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:45:41.961699 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-a143751e-5e5d-4753-9a17-df62558552a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:45:42.018568 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:45:42.094934 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 674/1359] 10.222.0.52 () {58 vars in 1059 bytes} [Thu Jul 21 22:45:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:45:42.099839 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad698f5-9a4d-4572-b3ea-707d78d05696 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:45:42.101423 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-0ad698f5-9a4d-4572-b3ea-707d78d05696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:45:42.258377 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad698f5-9a4d-4572-b3ea-707d78d05696 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:45:42.335073 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:45:42.379230 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3ffe60-bf12-424c-9ca4-c8647cba465d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bfdb655a2a44b33ab724daaf73a294b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:45:42.379882 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ffe60-bf12-424c-9ca4-c8647cba465d service nova] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:42.380310 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ffe60-bf12-424c-9ca4-c8647cba465d service nova] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:42.380595 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ffe60-bf12-424c-9ca4-c8647cba465d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:42.382233 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-6e3ffe60-bf12-424c-9ca4-c8647cba465d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:45:42.479850 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3ffe60-bf12-424c-9ca4-c8647cba465d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:45:42.482032 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3ffe60-bf12-424c-9ca4-c8647cba465d service nova] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:45:42.505653 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 686/1360] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:45:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:45:44.787755 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277adade-183e-4c9e-af10-7a7df1a83c57 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:44.787755 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277adade-183e-4c9e-af10-7a7df1a83c57 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:44.787755 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277adade-183e-4c9e-af10-7a7df1a83c57 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:44.787755 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 675/1361] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:45:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:45:44.790019 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b07cae-5f1f-4ee7-b29f-8ec64dd12789 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:44.790395 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b07cae-5f1f-4ee7-b29f-8ec64dd12789 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:44.790587 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b07cae-5f1f-4ee7-b29f-8ec64dd12789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:44.860922 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-c7b07cae-5f1f-4ee7-b29f-8ec64dd12789 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:45:44.862231 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-c7b07cae-5f1f-4ee7-b29f-8ec64dd12789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:45:44.905839 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:45:44.976402 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 687/1362] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:45:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:45:45.021650 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277adade-183e-4c9e-af10-7a7df1a83c57 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:45:45.023054 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-277adade-183e-4c9e-af10-7a7df1a83c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:45:45.182660 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277adade-183e-4c9e-af10-7a7df1a83c57 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:45:45.192557 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:45:45.312753 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1298009-376d-497b-82e8-c62bc1d4c9d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bfdb655a2a44b33ab724daaf73a294b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:45:45.313619 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1298009-376d-497b-82e8-c62bc1d4c9d8 service nova] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:45.314368 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1298009-376d-497b-82e8-c62bc1d4c9d8 service nova] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:45.314818 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1298009-376d-497b-82e8-c62bc1d4c9d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:45.316671 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-a1298009-376d-497b-82e8-c62bc1d4c9d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:45:45.412730 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1298009-376d-497b-82e8-c62bc1d4c9d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:45:45.414418 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1298009-376d-497b-82e8-c62bc1d4c9d8 service nova] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:45:45.433570 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 676/1363] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:45:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:45:47.929536 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1dfc32-7c17-4fe2-ae3c-4bec634fb69f None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:47.930104 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1dfc32-7c17-4fe2-ae3c-4bec634fb69f None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:47.930104 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1dfc32-7c17-4fe2-ae3c-4bec634fb69f None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:47.931509 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 688/1364] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:45:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:45:47.937387 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea0d056-7ad1-4ef2-a8dd-27207844058c None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:47.937814 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea0d056-7ad1-4ef2-a8dd-27207844058c None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:47.938007 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea0d056-7ad1-4ef2-a8dd-27207844058c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:47.953517 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:45:48.006360 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-eea0d056-7ad1-4ef2-a8dd-27207844058c None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:45:48.007617 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-eea0d056-7ad1-4ef2-a8dd-27207844058c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:45:48.118384 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 677/1365] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:45:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:45:48.165604 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea30e7c-d4ac-46b6-a190-f58b05071451 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:48.165895 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea30e7c-d4ac-46b6-a190-f58b05071451 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:48.166375 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea30e7c-d4ac-46b6-a190-f58b05071451 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:48.167773 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 689/1366] 10.222.0.52 () {54 vars in 962 bytes} [Thu Jul 21 22:45:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:45:48.174565 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf120031-7b6f-4dcc-83b0-c9b2df3c2829 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:48.174960 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf120031-7b6f-4dcc-83b0-c9b2df3c2829 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:48.175368 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf120031-7b6f-4dcc-83b0-c9b2df3c2829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:48.260319 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-bf120031-7b6f-4dcc-83b0-c9b2df3c2829 None None] MFA Rules not processed for user `6bfdb655a2a44b33ab724daaf73a294b`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:45:48.260319 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-bf120031-7b6f-4dcc-83b0-c9b2df3c2829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:45:48.298804 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:45:48.392461 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 678/1367] 10.222.0.52 () {58 vars in 1059 bytes} [Thu Jul 21 22:45:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:45:48.400448 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea30e7c-d4ac-46b6-a190-f58b05071451 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:45:48.402266 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-bea30e7c-d4ac-46b6-a190-f58b05071451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:45:48.562333 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea30e7c-d4ac-46b6-a190-f58b05071451 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:45:48.584214 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:45:48.686420 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349334f6-f36c-4af7-9262-97d9471538d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bfdb655a2a44b33ab724daaf73a294b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:45:48.687202 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349334f6-f36c-4af7-9262-97d9471538d3 service nova] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:48.687556 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349334f6-f36c-4af7-9262-97d9471538d3 service nova] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:48.687903 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349334f6-f36c-4af7-9262-97d9471538d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:48.689645 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-349334f6-f36c-4af7-9262-97d9471538d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:45:48.791422 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349334f6-f36c-4af7-9262-97d9471538d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:45:48.793430 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349334f6-f36c-4af7-9262-97d9471538d3 service nova] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:45:48.813795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 690/1368] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:45:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:45:49.711612 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb81596e-0efe-4b51-9914-adcaf99f99b4 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:49.712154 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb81596e-0efe-4b51-9914-adcaf99f99b4 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:49.712154 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb81596e-0efe-4b51-9914-adcaf99f99b4 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:49.713707 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 679/1369] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:45:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:45:49.718808 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89e5c8-f034-41e1-9c3e-d79aa32afde1 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:49.719250 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89e5c8-f034-41e1-9c3e-d79aa32afde1 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:49.719455 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89e5c8-f034-41e1-9c3e-d79aa32afde1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:49.751335 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:45:49.786500 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-5c89e5c8-f034-41e1-9c3e-d79aa32afde1 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:45:49.787754 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-5c89e5c8-f034-41e1-9c3e-d79aa32afde1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:45:49.893133 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 691/1370] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:45:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:45:50.238549 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705f588-1231-4bc7-a7c3-366cf30fc8c9 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:50.239384 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705f588-1231-4bc7-a7c3-366cf30fc8c9 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:50.239384 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705f588-1231-4bc7-a7c3-366cf30fc8c9 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:50.241491 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 680/1371] 10.222.0.52 () {54 vars in 964 bytes} [Thu Jul 21 22:45:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:45:50.246594 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c337068-d1d4-4e05-9bf2-9efcb4cbfabd None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:50.247027 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c337068-d1d4-4e05-9bf2-9efcb4cbfabd None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:50.247416 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c337068-d1d4-4e05-9bf2-9efcb4cbfabd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:50.324481 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-7c337068-d1d4-4e05-9bf2-9efcb4cbfabd None None] MFA Rules not processed for user `150f42651f754fda83f25fe3b408bce9`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:45:50.325834 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-7c337068-d1d4-4e05-9bf2-9efcb4cbfabd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:45:50.373870 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:45:50.404663 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 692/1372] 10.222.0.52 () {58 vars in 1061 bytes} [Thu Jul 21 22:45:50 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:45:50.412390 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705f588-1231-4bc7-a7c3-366cf30fc8c9 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:45:50.413890 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-1705f588-1231-4bc7-a7c3-366cf30fc8c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:45:50.532711 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705f588-1231-4bc7-a7c3-366cf30fc8c9 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:45:50.575582 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:45:50.611920 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80592bca-4af0-477d-ad72-887844b247d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:45:50.612566 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80592bca-4af0-477d-ad72-887844b247d7 service cinder] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:50.612845 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80592bca-4af0-477d-ad72-887844b247d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:50.613118 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80592bca-4af0-477d-ad72-887844b247d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:50.614802 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-80592bca-4af0-477d-ad72-887844b247d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:45:50.729800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80592bca-4af0-477d-ad72-887844b247d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:45:50.731333 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80592bca-4af0-477d-ad72-887844b247d7 service cinder] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:45:50.751607 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 681/1373] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:45:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:45:51.070320 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea95c80-2944-484b-81f3-05613077a777 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:51.070921 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea95c80-2944-484b-81f3-05613077a777 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:51.070921 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea95c80-2944-484b-81f3-05613077a777 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:51.075398 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 693/1374] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:45:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:45:51.078040 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f7551-6911-4e8b-a2ea-8bf64f70728d None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:51.078444 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f7551-6911-4e8b-a2ea-8bf64f70728d None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:51.078873 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f7551-6911-4e8b-a2ea-8bf64f70728d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:51.166763 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-310f7551-6911-4e8b-a2ea-8bf64f70728d None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:45:51.168985 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-310f7551-6911-4e8b-a2ea-8bf64f70728d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:45:51.203019 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:45:51.307429 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 682/1375] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:45:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:45:51.351307 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea95c80-2944-484b-81f3-05613077a777 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:45:51.353214 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-1ea95c80-2944-484b-81f3-05613077a777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:45:51.511643 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea95c80-2944-484b-81f3-05613077a777 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:45:51.564157 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:45:51.626124 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5882e2e4-6385-429c-9869-ec16a7c587ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bfdb655a2a44b33ab724daaf73a294b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:45:51.626788 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882e2e4-6385-429c-9869-ec16a7c587ce service nova] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:51.627103 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882e2e4-6385-429c-9869-ec16a7c587ce service nova] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:51.627374 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882e2e4-6385-429c-9869-ec16a7c587ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:51.628992 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-5882e2e4-6385-429c-9869-ec16a7c587ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:45:51.719188 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5882e2e4-6385-429c-9869-ec16a7c587ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:45:51.720779 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5882e2e4-6385-429c-9869-ec16a7c587ce service nova] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:45:51.739061 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 694/1376] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:45:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:45:53.955185 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae2e6f-1746-4470-b2c8-3f65a4ebb57b None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:53.955702 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae2e6f-1746-4470-b2c8-3f65a4ebb57b None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:53.955798 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae2e6f-1746-4470-b2c8-3f65a4ebb57b None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:53.957148 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 683/1377] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:45:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:45:53.962445 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4be2bb-f090-4981-91e0-7c25fffdc3d1 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:53.962886 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4be2bb-f090-4981-91e0-7c25fffdc3d1 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:53.963047 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4be2bb-f090-4981-91e0-7c25fffdc3d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:54.027468 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-6e4be2bb-f090-4981-91e0-7c25fffdc3d1 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:45:54.029192 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-6e4be2bb-f090-4981-91e0-7c25fffdc3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:45:54.035862 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:45:54.134629 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 695/1378] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:45:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:45:54.179230 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae2e6f-1746-4470-b2c8-3f65a4ebb57b None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:45:54.180889 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-93ae2e6f-1746-4470-b2c8-3f65a4ebb57b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:45:54.376280 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae2e6f-1746-4470-b2c8-3f65a4ebb57b None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:45:54.380067 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:45:54.506742 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563db2da-2c62-4af0-a603-9882852f73ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bfdb655a2a44b33ab724daaf73a294b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:45:54.507430 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563db2da-2c62-4af0-a603-9882852f73ba service nova] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:54.507715 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563db2da-2c62-4af0-a603-9882852f73ba service nova] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:54.507994 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563db2da-2c62-4af0-a603-9882852f73ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:54.509695 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-563db2da-2c62-4af0-a603-9882852f73ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:45:54.619765 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563db2da-2c62-4af0-a603-9882852f73ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:45:54.622608 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563db2da-2c62-4af0-a603-9882852f73ba service nova] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:45:54.643406 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 684/1379] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:45:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:45:56.812804 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b03c65-9de0-4aab-9bfc-68dddcff183c None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:56.813326 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b03c65-9de0-4aab-9bfc-68dddcff183c None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:56.813326 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b03c65-9de0-4aab-9bfc-68dddcff183c None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:56.814792 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 696/1380] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:45:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:45:56.819847 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0b4ba7-6c08-4fc1-afcb-a9361f632fcb None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:56.820242 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0b4ba7-6c08-4fc1-afcb-a9361f632fcb None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:56.820410 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0b4ba7-6c08-4fc1-afcb-a9361f632fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:56.835460 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:45:56.893737 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-7e0b4ba7-6c08-4fc1-afcb-a9361f632fcb None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:45:56.895045 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-7e0b4ba7-6c08-4fc1-afcb-a9361f632fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:45:57.001139 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 685/1381] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:45:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:45:57.045745 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b03c65-9de0-4aab-9bfc-68dddcff183c None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:45:57.047286 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-f4b03c65-9de0-4aab-9bfc-68dddcff183c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:45:57.194342 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:45:57.208340 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b03c65-9de0-4aab-9bfc-68dddcff183c None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:45:57.338270 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbe7b5a-a662-4bb9-997e-ed576f3e100e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bfdb655a2a44b33ab724daaf73a294b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:45:57.339123 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe7b5a-a662-4bb9-997e-ed576f3e100e service nova] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:57.339225 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe7b5a-a662-4bb9-997e-ed576f3e100e service nova] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:57.339489 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe7b5a-a662-4bb9-997e-ed576f3e100e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:57.341182 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-fcbe7b5a-a662-4bb9-997e-ed576f3e100e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:45:57.440984 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbe7b5a-a662-4bb9-997e-ed576f3e100e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:45:57.443350 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbe7b5a-a662-4bb9-997e-ed576f3e100e service nova] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:45:57.457616 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:45:57.467993 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 697/1382] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:45:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:45:59.992159 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb19b8c-0655-4f1d-bcd8-b23d85a85e8f None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:45:59.992748 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb19b8c-0655-4f1d-bcd8-b23d85a85e8f None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:45:59.992748 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb19b8c-0655-4f1d-bcd8-b23d85a85e8f None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:45:59.994132 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 686/1383] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:45:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:46:00.000466 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b8264-b36c-44fc-990d-efae92e8b248 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:00.000968 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b8264-b36c-44fc-990d-efae92e8b248 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:00.001151 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b8264-b36c-44fc-990d-efae92e8b248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:00.070802 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-f91b8264-b36c-44fc-990d-efae92e8b248 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:46:00.072949 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-f91b8264-b36c-44fc-990d-efae92e8b248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:00.181077 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 698/1384] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:45:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:46:00.236186 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb19b8c-0655-4f1d-bcd8-b23d85a85e8f None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:46:00.237831 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-6fb19b8c-0655-4f1d-bcd8-b23d85a85e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:00.328161 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:46:00.392095 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb19b8c-0655-4f1d-bcd8-b23d85a85e8f None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:46:00.515153 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107cca2e-af20-48ef-ace9-c1aa292c211b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bfdb655a2a44b33ab724daaf73a294b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:46:00.515851 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107cca2e-af20-48ef-ace9-c1aa292c211b service nova] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:00.516169 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107cca2e-af20-48ef-ace9-c1aa292c211b service nova] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:00.516467 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107cca2e-af20-48ef-ace9-c1aa292c211b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:00.518252 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-107cca2e-af20-48ef-ace9-c1aa292c211b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:00.570924 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:46:00.618963 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107cca2e-af20-48ef-ace9-c1aa292c211b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:46:00.621048 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107cca2e-af20-48ef-ace9-c1aa292c211b service nova] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:46:00.644902 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 687/1385] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:46:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:46:02.734319 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f0d1a-36e4-4d0b-82a6-9d9f269092e0 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:02.734874 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f0d1a-36e4-4d0b-82a6-9d9f269092e0 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:02.735086 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f0d1a-36e4-4d0b-82a6-9d9f269092e0 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:02.736913 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 699/1386] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:46:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:46:02.742199 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349e98e-13af-4734-a874-ea44a265b978 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:02.742632 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349e98e-13af-4734-a874-ea44a265b978 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:02.742846 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349e98e-13af-4734-a874-ea44a265b978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:02.808558 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-e349e98e-13af-4734-a874-ea44a265b978 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:46:02.809787 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-e349e98e-13af-4734-a874-ea44a265b978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:02.925401 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 688/1387] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:46:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:46:02.970563 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f0d1a-36e4-4d0b-82a6-9d9f269092e0 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:46:02.972640 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-361f0d1a-36e4-4d0b-82a6-9d9f269092e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:03.055465 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:46:03.136915 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f0d1a-36e4-4d0b-82a6-9d9f269092e0 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:46:03.269261 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349e4eb7-6569-4c25-b687-5a79d5670fd2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bfdb655a2a44b33ab724daaf73a294b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:46:03.269914 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e4eb7-6569-4c25-b687-5a79d5670fd2 service nova] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:03.270201 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e4eb7-6569-4c25-b687-5a79d5670fd2 service nova] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:03.270478 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e4eb7-6569-4c25-b687-5a79d5670fd2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:03.272147 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-349e4eb7-6569-4c25-b687-5a79d5670fd2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:03.316750 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:46:03.372069 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349e4eb7-6569-4c25-b687-5a79d5670fd2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:46:03.373674 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349e4eb7-6569-4c25-b687-5a79d5670fd2 service nova] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:46:03.391584 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 700/1388] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:46:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:46:05.579846 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44011f2-2fb4-476e-b7e8-f26c0e4335b8 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:05.580407 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44011f2-2fb4-476e-b7e8-f26c0e4335b8 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:05.580407 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44011f2-2fb4-476e-b7e8-f26c0e4335b8 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:05.581924 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 689/1389] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:46:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:46:05.587065 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b0a57-b82f-4ccb-b7fc-b7344e74d22c None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:05.587507 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b0a57-b82f-4ccb-b7fc-b7344e74d22c None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:05.587604 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b0a57-b82f-4ccb-b7fc-b7344e74d22c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:05.652952 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-c95b0a57-b82f-4ccb-b7fc-b7344e74d22c None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:46:05.654159 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-c95b0a57-b82f-4ccb-b7fc-b7344e74d22c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:05.752211 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 701/1390] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:46:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:46:05.796240 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44011f2-2fb4-476e-b7e8-f26c0e4335b8 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:46:05.797835 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-f44011f2-2fb4-476e-b7e8-f26c0e4335b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:05.829896 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:46:05.949706 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44011f2-2fb4-476e-b7e8-f26c0e4335b8 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:46:06.050794 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:46:06.061962 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3858e2-1e01-4169-82e6-992cbc0ce9db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bfdb655a2a44b33ab724daaf73a294b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:46:06.062617 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3858e2-1e01-4169-82e6-992cbc0ce9db service nova] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:06.062937 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3858e2-1e01-4169-82e6-992cbc0ce9db service nova] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:06.063266 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3858e2-1e01-4169-82e6-992cbc0ce9db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:06.064884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-fa3858e2-1e01-4169-82e6-992cbc0ce9db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:06.112533 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc33087f-273c-4c5e-bc57-7a87bfdb7f1d None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:06.112832 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc33087f-273c-4c5e-bc57-7a87bfdb7f1d None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:06.113115 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc33087f-273c-4c5e-bc57-7a87bfdb7f1d None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:06.114484 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 702/1391] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:46:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:46:06.119275 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b4570-fcec-4c33-8356-afc564895618 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:06.119567 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b4570-fcec-4c33-8356-afc564895618 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:06.119848 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b4570-fcec-4c33-8356-afc564895618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:06.133683 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:46:06.157378 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3858e2-1e01-4169-82e6-992cbc0ce9db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:46:06.159008 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3858e2-1e01-4169-82e6-992cbc0ce9db service nova] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:46:06.177907 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 690/1392] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:46:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:46:06.186666 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-650b4570-fcec-4c33-8356-afc564895618 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:46:06.187936 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-650b4570-fcec-4c33-8356-afc564895618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:06.290475 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 703/1393] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:46:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:46:06.348067 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe6930-059f-47b4-9d20-a11607621900 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:06.348469 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe6930-059f-47b4-9d20-a11607621900 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:06.348837 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe6930-059f-47b4-9d20-a11607621900 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:06.350433 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 691/1394] 10.222.0.52 () {54 vars in 966 bytes} [Thu Jul 21 22:46:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:46:06.356591 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4928a0-ab51-402a-89a2-5d336e21c605 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:06.356884 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4928a0-ab51-402a-89a2-5d336e21c605 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:06.357168 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4928a0-ab51-402a-89a2-5d336e21c605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:06.442104 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-ff4928a0-ab51-402a-89a2-5d336e21c605 None None] MFA Rules not processed for user `927778d005fd48a4b4b8fd3c4419bb36`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:46:06.443388 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-ff4928a0-ab51-402a-89a2-5d336e21c605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:06.481823 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:46:06.525960 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 704/1395] 10.222.0.52 () {58 vars in 1063 bytes} [Thu Jul 21 22:46:06 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:46:06.533837 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fe6930-059f-47b4-9d20-a11607621900 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:46:06.535382 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-c8fe6930-059f-47b4-9d20-a11607621900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:06.642088 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fe6930-059f-47b4-9d20-a11607621900 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:46:06.655780 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:46:06.716829 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4828121a-83f7-4f67-bae9-bedf44665bd5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '927778d005fd48a4b4b8fd3c4419bb36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:46:06.717481 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4828121a-83f7-4f67-bae9-bedf44665bd5 service glance] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:06.717760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4828121a-83f7-4f67-bae9-bedf44665bd5 service glance] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:06.718033 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4828121a-83f7-4f67-bae9-bedf44665bd5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:06.719715 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-4828121a-83f7-4f67-bae9-bedf44665bd5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:06.829942 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4828121a-83f7-4f67-bae9-bedf44665bd5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:46:06.831459 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4828121a-83f7-4f67-bae9-bedf44665bd5 service glance] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:46:06.850283 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 692/1396] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:46:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:46:07.692056 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d07d87-8562-4a02-be61-cc4b09ab83ce None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:07.692539 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d07d87-8562-4a02-be61-cc4b09ab83ce None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:07.692631 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d07d87-8562-4a02-be61-cc4b09ab83ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:07.757398 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-33d07d87-8562-4a02-be61-cc4b09ab83ce None None] MFA Rules not processed for user `63cc904578554aea84ff1917a5ed075e`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:46:07.758606 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-33d07d87-8562-4a02-be61-cc4b09ab83ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:07.854504 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 705/1397] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jul 21 22:46:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:46:07.900465 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f6f29-62f7-427d-8ed9-0916e54e0f10 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:07.900982 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f6f29-62f7-427d-8ed9-0916e54e0f10 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:07.901232 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f6f29-62f7-427d-8ed9-0916e54e0f10 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:07.903452 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 693/1398] 10.222.0.52 () {54 vars in 943 bytes} [Thu Jul 21 22:46:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:46:07.910465 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f6fa4-1100-47bd-892a-04497d301ee9 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:07.910793 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f6fa4-1100-47bd-892a-04497d301ee9 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:07.911068 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f6fa4-1100-47bd-892a-04497d301ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:07.980879 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:46:07.999692 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-619f6fa4-1100-47bd-892a-04497d301ee9 None None] MFA Rules not processed for user `ba7eef4e1b27476ab4016449d9c3a4b9`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:46:08.000899 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-619f6fa4-1100-47bd-892a-04497d301ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:08.109315 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 706/1399] 10.222.0.52 () {58 vars in 1040 bytes} [Thu Jul 21 22:46:07 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:46:08.118722 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369f6f29-62f7-427d-8ed9-0916e54e0f10 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:46:08.120291 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-369f6f29-62f7-427d-8ed9-0916e54e0f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:08.166477 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:46:08.376630 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369f6f29-62f7-427d-8ed9-0916e54e0f10 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:46:08.381134 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02368f-1f08-4e2b-b4c0-5028f63223a5 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:08.381426 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02368f-1f08-4e2b-b4c0-5028f63223a5 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:08.381707 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02368f-1f08-4e2b-b4c0-5028f63223a5 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:08.383154 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 707/1400] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:46:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:46:08.388265 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd27714-dd0d-4da9-8bce-6dbb1eb129cc None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:08.388549 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd27714-dd0d-4da9-8bce-6dbb1eb129cc None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:08.388826 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd27714-dd0d-4da9-8bce-6dbb1eb129cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:08.453146 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-dcd27714-dd0d-4da9-8bce-6dbb1eb129cc None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:46:08.454353 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-dcd27714-dd0d-4da9-8bce-6dbb1eb129cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:08.481756 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916117f9-8e40-404c-b90a-869d8ae64adf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba7eef4e1b27476ab4016449d9c3a4b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:46:08.482414 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916117f9-8e40-404c-b90a-869d8ae64adf service swift] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:08.482694 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916117f9-8e40-404c-b90a-869d8ae64adf service swift] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:08.483001 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916117f9-8e40-404c-b90a-869d8ae64adf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:08.484700 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-916117f9-8e40-404c-b90a-869d8ae64adf service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:08.518507 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:46:08.519880 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:46:08.561800 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 708/1401] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:46:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:46:08.574322 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916117f9-8e40-404c-b90a-869d8ae64adf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:46:08.575957 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916117f9-8e40-404c-b90a-869d8ae64adf service swift] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:46:08.583994 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 694/1402] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jul 21 22:46:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 22:46:08.607735 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd27714-dd0d-4da9-8bce-6dbb1eb129cc None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:46:08.609374 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-dcd27714-dd0d-4da9-8bce-6dbb1eb129cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:08.727297 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd27714-dd0d-4da9-8bce-6dbb1eb129cc None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:46:08.806674 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:46:08.862222 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b421b67-334a-4231-917b-a0f9a5787371 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bfdb655a2a44b33ab724daaf73a294b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:46:08.862912 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b421b67-334a-4231-917b-a0f9a5787371 service nova] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:08.863196 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b421b67-334a-4231-917b-a0f9a5787371 service nova] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:08.863468 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b421b67-334a-4231-917b-a0f9a5787371 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:08.865135 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-2b421b67-334a-4231-917b-a0f9a5787371 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:08.962165 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b421b67-334a-4231-917b-a0f9a5787371 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:46:08.963792 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b421b67-334a-4231-917b-a0f9a5787371 service nova] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:46:08.985220 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 709/1403] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:46:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:46:11.247398 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f131af1-4a1e-496e-8fef-437c1147cb9d None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:11.248322 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f131af1-4a1e-496e-8fef-437c1147cb9d None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:11.250727 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f131af1-4a1e-496e-8fef-437c1147cb9d None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:11.250727 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 695/1404] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:46:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:46:11.255750 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2284499-4dc5-4fa9-9092-503b8c982443 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:11.256198 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2284499-4dc5-4fa9-9092-503b8c982443 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:11.256293 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2284499-4dc5-4fa9-9092-503b8c982443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:11.330797 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-d2284499-4dc5-4fa9-9092-503b8c982443 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:46:11.332037 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-d2284499-4dc5-4fa9-9092-503b8c982443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:11.369828 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:46:11.445403 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 710/1405] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:46:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:46:11.490295 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f131af1-4a1e-496e-8fef-437c1147cb9d None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:46:11.492088 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-1f131af1-4a1e-496e-8fef-437c1147cb9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:11.666786 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f131af1-4a1e-496e-8fef-437c1147cb9d None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:46:11.703337 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:46:11.820514 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9c0657-9453-4ba6-aea5-817629861fa8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bfdb655a2a44b33ab724daaf73a294b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:46:11.821570 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9c0657-9453-4ba6-aea5-817629861fa8 service nova] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:11.822071 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9c0657-9453-4ba6-aea5-817629861fa8 service nova] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:11.822528 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9c0657-9453-4ba6-aea5-817629861fa8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:11.824370 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-de9c0657-9453-4ba6-aea5-817629861fa8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:11.931009 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9c0657-9453-4ba6-aea5-817629861fa8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:46:11.955579 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9c0657-9453-4ba6-aea5-817629861fa8 service nova] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:46:11.961692 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 696/1406] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:46:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:46:14.640350 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f322d-8fc4-4d21-abc7-6be54460f2d1 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:14.641363 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f322d-8fc4-4d21-abc7-6be54460f2d1 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:14.641946 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f322d-8fc4-4d21-abc7-6be54460f2d1 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:14.643664 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 711/1407] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:46:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:46:14.653041 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe43d2-5f40-42a0-898c-5cb492e1c124 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:14.653612 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe43d2-5f40-42a0-898c-5cb492e1c124 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:14.653612 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe43d2-5f40-42a0-898c-5cb492e1c124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:14.693853 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:46:14.726417 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-cdfe43d2-5f40-42a0-898c-5cb492e1c124 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:46:14.726417 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-cdfe43d2-5f40-42a0-898c-5cb492e1c124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:14.835375 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 697/1408] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:46:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:46:14.883999 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f322d-8fc4-4d21-abc7-6be54460f2d1 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:46:14.885674 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-721f322d-8fc4-4d21-abc7-6be54460f2d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:15.049405 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f322d-8fc4-4d21-abc7-6be54460f2d1 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:46:15.067739 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:46:15.175154 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2437f7-a255-40a4-ad99-ac1b59618062 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bfdb655a2a44b33ab724daaf73a294b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:46:15.175737 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2437f7-a255-40a4-ad99-ac1b59618062 service nova] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:15.176016 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2437f7-a255-40a4-ad99-ac1b59618062 service nova] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:15.176290 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2437f7-a255-40a4-ad99-ac1b59618062 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:15.177928 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-0a2437f7-a255-40a4-ad99-ac1b59618062 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:15.280988 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2437f7-a255-40a4-ad99-ac1b59618062 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:46:15.282619 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2437f7-a255-40a4-ad99-ac1b59618062 service nova] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:46:15.303326 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 712/1409] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:46:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:46:24.835532 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=53 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:46:25.300790 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=91 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:46:32.703676 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cac8a-519c-47aa-ba1a-2977be0433fd None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:32.704237 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cac8a-519c-47aa-ba1a-2977be0433fd None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:32.704237 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cac8a-519c-47aa-ba1a-2977be0433fd None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:32.705779 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 698/1410] 10.222.0.52 () {54 vars in 918 bytes} [Thu Jul 21 22:46:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:46:32.710806 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e294d-97aa-4cae-8e22-86a5a1e1cd48 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:32.711210 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e294d-97aa-4cae-8e22-86a5a1e1cd48 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:32.711391 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e294d-97aa-4cae-8e22-86a5a1e1cd48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:32.788386 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-f79e294d-97aa-4cae-8e22-86a5a1e1cd48 None None] MFA Rules not processed for user `10c0bed169344a77bfad29624c5e5a8e`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:46:32.789539 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-f79e294d-97aa-4cae-8e22-86a5a1e1cd48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:32.894917 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 713/1411] 10.222.0.52 () {58 vars in 1015 bytes} [Thu Jul 21 22:46:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:46:32.909959 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d968a3-8a48-48aa-917b-ddc227272813 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:32.910245 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d968a3-8a48-48aa-917b-ddc227272813 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:32.910519 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d968a3-8a48-48aa-917b-ddc227272813 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:32.911931 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 699/1412] 10.222.0.52 () {54 vars in 918 bytes} [Thu Jul 21 22:46:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:46:32.917124 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dcc325-ee73-44a4-b3b9-24ecec3db089 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:32.917413 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dcc325-ee73-44a4-b3b9-24ecec3db089 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:32.917694 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dcc325-ee73-44a4-b3b9-24ecec3db089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:32.988869 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:46:32.991836 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-24dcc325-ee73-44a4-b3b9-24ecec3db089 None None] MFA Rules not processed for user `6bfdb655a2a44b33ab724daaf73a294b`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:46:32.993034 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-24dcc325-ee73-44a4-b3b9-24ecec3db089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:33.110328 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 714/1413] 10.222.0.52 () {58 vars in 1015 bytes} [Thu Jul 21 22:46:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:46:35.602191 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d09504-ec0a-4441-9ef5-8ab5524f8ca3 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:35.602734 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d09504-ec0a-4441-9ef5-8ab5524f8ca3 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:35.602734 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d09504-ec0a-4441-9ef5-8ab5524f8ca3 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:35.604178 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 700/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 22:46:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:46:35.609471 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79ce3a-09e7-4acd-9847-4689b1c2ec9e None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:35.609863 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79ce3a-09e7-4acd-9847-4689b1c2ec9e None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:35.610024 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79ce3a-09e7-4acd-9847-4689b1c2ec9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:35.673140 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-8e79ce3a-09e7-4acd-9847-4689b1c2ec9e None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:46:35.674324 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-8e79ce3a-09e7-4acd-9847-4689b1c2ec9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:35.732084 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:46:35.773981 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 715/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 22:46:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:46:35.816847 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d09504-ec0a-4441-9ef5-8ab5524f8ca3 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:46:35.818228 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-89d09504-ec0a-4441-9ef5-8ab5524f8ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:35.962380 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d09504-ec0a-4441-9ef5-8ab5524f8ca3 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:46:36.035960 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:46:36.075649 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04746ba3-5e37-4ca2-abe6-a6b4d7d26c52 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bfdb655a2a44b33ab724daaf73a294b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:46:36.076295 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04746ba3-5e37-4ca2-abe6-a6b4d7d26c52 service nova] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:36.076577 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04746ba3-5e37-4ca2-abe6-a6b4d7d26c52 service nova] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:36.076854 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04746ba3-5e37-4ca2-abe6-a6b4d7d26c52 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:36.078536 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-04746ba3-5e37-4ca2-abe6-a6b4d7d26c52 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:36.167819 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04746ba3-5e37-4ca2-abe6-a6b4d7d26c52 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:46:36.169407 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04746ba3-5e37-4ca2-abe6-a6b4d7d26c52 service nova] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:46:36.187259 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 701/1416] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:46:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:46:37.079154 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f026c04-5265-4df6-8a53-4749edc550be None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:37.079705 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f026c04-5265-4df6-8a53-4749edc550be None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:37.079705 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f026c04-5265-4df6-8a53-4749edc550be None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:37.081066 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 716/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 22:46:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:46:37.086196 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd316b68-7195-4232-a19c-5590ee364385 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:37.086639 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd316b68-7195-4232-a19c-5590ee364385 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:37.086810 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd316b68-7195-4232-a19c-5590ee364385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:37.160707 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-dd316b68-7195-4232-a19c-5590ee364385 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:46:37.161825 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-dd316b68-7195-4232-a19c-5590ee364385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:37.188394 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:46:37.257379 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 702/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 22:46:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:46:37.406326 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f026c04-5265-4df6-8a53-4749edc550be None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:46:37.407868 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-0f026c04-5265-4df6-8a53-4749edc550be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:37.553745 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f026c04-5265-4df6-8a53-4749edc550be None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:46:37.593491 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:46:37.665826 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a1918b-d38d-4d4e-88eb-bea2f0355c21 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bfdb655a2a44b33ab724daaf73a294b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:46:37.667502 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1918b-d38d-4d4e-88eb-bea2f0355c21 service nova] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:37.668018 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1918b-d38d-4d4e-88eb-bea2f0355c21 service nova] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:37.668522 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1918b-d38d-4d4e-88eb-bea2f0355c21 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:37.671756 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-56a1918b-d38d-4d4e-88eb-bea2f0355c21 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:37.761574 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a1918b-d38d-4d4e-88eb-bea2f0355c21 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:46:37.763114 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a1918b-d38d-4d4e-88eb-bea2f0355c21 service nova] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:46:37.781376 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 717/1419] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:46:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:46:38.660028 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaefb86-7d67-4eb4-95eb-6648f69a0896 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:38.660589 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaefb86-7d67-4eb4-95eb-6648f69a0896 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:38.660589 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaefb86-7d67-4eb4-95eb-6648f69a0896 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:38.661972 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 703/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 22:46:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:46:38.667026 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31406255-1e7f-4455-97f0-eb2e11e905b3 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:38.667456 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31406255-1e7f-4455-97f0-eb2e11e905b3 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:38.667623 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31406255-1e7f-4455-97f0-eb2e11e905b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:38.725040 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:46:38.747056 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-31406255-1e7f-4455-97f0-eb2e11e905b3 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:46:38.748276 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-31406255-1e7f-4455-97f0-eb2e11e905b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:38.846815 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 718/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 22:46:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:46:38.997598 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaefb86-7d67-4eb4-95eb-6648f69a0896 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:46:38.999065 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-fdaefb86-7d67-4eb4-95eb-6648f69a0896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:39.156054 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaefb86-7d67-4eb4-95eb-6648f69a0896 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:46:39.168590 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:46:39.268654 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c923c90-2686-49de-a2f6-b3dc2833c9ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bfdb655a2a44b33ab724daaf73a294b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:46:39.269228 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c923c90-2686-49de-a2f6-b3dc2833c9ad service nova] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:39.269538 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c923c90-2686-49de-a2f6-b3dc2833c9ad service nova] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:39.269811 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c923c90-2686-49de-a2f6-b3dc2833c9ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:39.271520 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-4c923c90-2686-49de-a2f6-b3dc2833c9ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:39.360201 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c923c90-2686-49de-a2f6-b3dc2833c9ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:46:39.361789 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c923c90-2686-49de-a2f6-b3dc2833c9ad service nova] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:46:39.379572 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 704/1422] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:46:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:46:40.247970 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8a466-14f5-4d5b-9f70-c1e8fd866eed None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:40.248551 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8a466-14f5-4d5b-9f70-c1e8fd866eed None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:40.248551 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8a466-14f5-4d5b-9f70-c1e8fd866eed None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:40.249881 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 719/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 22:46:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:46:40.255115 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1145d023-30c2-4c10-ba65-7708f3ca5786 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:40.255555 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1145d023-30c2-4c10-ba65-7708f3ca5786 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:40.255665 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1145d023-30c2-4c10-ba65-7708f3ca5786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:40.274160 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:46:40.328943 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-1145d023-30c2-4c10-ba65-7708f3ca5786 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:46:40.330133 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-1145d023-30c2-4c10-ba65-7708f3ca5786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:40.427038 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 705/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 22:46:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:46:40.576207 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d8a466-14f5-4d5b-9f70-c1e8fd866eed None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:46:40.577792 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-95d8a466-14f5-4d5b-9f70-c1e8fd866eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:40.687829 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:46:40.723502 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d8a466-14f5-4d5b-9f70-c1e8fd866eed None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:46:40.838362 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eee0095-88fb-4a8a-b77d-86a24008d1e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bfdb655a2a44b33ab724daaf73a294b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:46:40.839067 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee0095-88fb-4a8a-b77d-86a24008d1e4 service nova] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:40.839424 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee0095-88fb-4a8a-b77d-86a24008d1e4 service nova] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:40.839639 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee0095-88fb-4a8a-b77d-86a24008d1e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:40.841300 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-6eee0095-88fb-4a8a-b77d-86a24008d1e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:40.917303 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:46:40.930918 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eee0095-88fb-4a8a-b77d-86a24008d1e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:46:40.932519 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eee0095-88fb-4a8a-b77d-86a24008d1e4 service nova] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:46:40.951034 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 720/1425] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:46:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:46:41.811856 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edcf7f5-1d27-4504-9c48-f142b8b68a1e None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:41.812382 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edcf7f5-1d27-4504-9c48-f142b8b68a1e None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:41.812492 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edcf7f5-1d27-4504-9c48-f142b8b68a1e None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:41.814099 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 706/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 22:46:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:46:41.819276 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4401dd2-ac1e-4f5f-a8be-6d15e2373fcd None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:41.819696 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4401dd2-ac1e-4f5f-a8be-6d15e2373fcd None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:41.819830 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4401dd2-ac1e-4f5f-a8be-6d15e2373fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:41.894557 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-e4401dd2-ac1e-4f5f-a8be-6d15e2373fcd None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:46:41.895798 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-e4401dd2-ac1e-4f5f-a8be-6d15e2373fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:41.995051 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 721/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 22:46:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:46:42.145561 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edcf7f5-1d27-4504-9c48-f142b8b68a1e None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:46:42.147215 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-9edcf7f5-1d27-4504-9c48-f142b8b68a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:42.231626 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:46:42.292600 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edcf7f5-1d27-4504-9c48-f142b8b68a1e None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:46:42.404937 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e217035f-6002-4307-99c4-345e5b61fb49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bfdb655a2a44b33ab724daaf73a294b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:46:42.405501 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e217035f-6002-4307-99c4-345e5b61fb49 service nova] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:42.405780 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e217035f-6002-4307-99c4-345e5b61fb49 service nova] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:42.406051 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e217035f-6002-4307-99c4-345e5b61fb49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:42.407727 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-e217035f-6002-4307-99c4-345e5b61fb49 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:42.454838 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:46:42.496450 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e217035f-6002-4307-99c4-345e5b61fb49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:46:42.498055 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e217035f-6002-4307-99c4-345e5b61fb49 service nova] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:46:42.516920 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 707/1428] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:46:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:46:43.363012 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf83714-d89a-47ae-b58a-c88282e8919c None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:43.363628 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf83714-d89a-47ae-b58a-c88282e8919c None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:43.363628 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf83714-d89a-47ae-b58a-c88282e8919c None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:43.364993 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 722/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 22:46:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:46:43.370090 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9358cd-37af-4e18-8fb9-310a2c125baa None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:43.370492 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9358cd-37af-4e18-8fb9-310a2c125baa None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:43.370665 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9358cd-37af-4e18-8fb9-310a2c125baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:43.449577 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-3f9358cd-37af-4e18-8fb9-310a2c125baa None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:46:43.450805 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-3f9358cd-37af-4e18-8fb9-310a2c125baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:43.547007 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 708/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 22:46:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:46:43.699394 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf83714-d89a-47ae-b58a-c88282e8919c None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:46:43.700851 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-1bf83714-d89a-47ae-b58a-c88282e8919c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:43.749597 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:46:43.848149 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf83714-d89a-47ae-b58a-c88282e8919c None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:46:43.960142 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9756ec40-48e1-4bd1-9a4a-e5d643493fca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bfdb655a2a44b33ab724daaf73a294b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:46:43.960721 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9756ec40-48e1-4bd1-9a4a-e5d643493fca service nova] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:43.961008 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9756ec40-48e1-4bd1-9a4a-e5d643493fca service nova] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:43.961286 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9756ec40-48e1-4bd1-9a4a-e5d643493fca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:43.963044 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-9756ec40-48e1-4bd1-9a4a-e5d643493fca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:43.970167 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:46:44.052685 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9756ec40-48e1-4bd1-9a4a-e5d643493fca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:46:44.054264 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9756ec40-48e1-4bd1-9a4a-e5d643493fca service nova] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:46:44.072794 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 723/1431] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:46:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:46:44.945148 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9adbb2d-2727-497a-9d5e-6ed445535325 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:44.945611 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9adbb2d-2727-497a-9d5e-6ed445535325 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:44.945760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9adbb2d-2727-497a-9d5e-6ed445535325 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:44.947816 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 709/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 22:46:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:46:44.952709 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4752c-20ff-4c2d-9820-0f174dea74d7 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:44.953144 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4752c-20ff-4c2d-9820-0f174dea74d7 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:44.953273 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4752c-20ff-4c2d-9820-0f174dea74d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:45.028959 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-2da4752c-20ff-4c2d-9820-0f174dea74d7 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:46:45.030173 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-2da4752c-20ff-4c2d-9820-0f174dea74d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:45.096819 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:46:45.129198 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 724/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 22:46:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:46:45.279219 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9adbb2d-2727-497a-9d5e-6ed445535325 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:46:45.280678 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-b9adbb2d-2727-497a-9d5e-6ed445535325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:45.298630 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:46:45.425218 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9adbb2d-2727-497a-9d5e-6ed445535325 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:46:45.514193 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:46:45.540768 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547b6423-7094-45cc-8825-3804c5edc028 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bfdb655a2a44b33ab724daaf73a294b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:46:45.541356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b6423-7094-45cc-8825-3804c5edc028 service nova] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:45.541689 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b6423-7094-45cc-8825-3804c5edc028 service nova] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:45.541966 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b6423-7094-45cc-8825-3804c5edc028 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:45.543744 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-547b6423-7094-45cc-8825-3804c5edc028 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:45.632766 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547b6423-7094-45cc-8825-3804c5edc028 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:46:45.634901 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547b6423-7094-45cc-8825-3804c5edc028 service nova] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:46:45.653346 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 710/1434] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:46:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:46:46.506652 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e54b8b-77b5-431f-9003-07e5205db643 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:46.507143 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e54b8b-77b5-431f-9003-07e5205db643 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:46.507255 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e54b8b-77b5-431f-9003-07e5205db643 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:46.508609 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 725/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 22:46:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:46:46.514128 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caef557-df8a-49e7-98f7-8066db2fd532 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:46.514524 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caef557-df8a-49e7-98f7-8066db2fd532 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:46.514671 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caef557-df8a-49e7-98f7-8066db2fd532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:46.588337 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-7caef557-df8a-49e7-98f7-8066db2fd532 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:46:46.589551 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-7caef557-df8a-49e7-98f7-8066db2fd532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:46.630849 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:46:46.686246 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 711/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 22:46:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:46:46.835208 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e54b8b-77b5-431f-9003-07e5205db643 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:46:46.836744 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-f7e54b8b-77b5-431f-9003-07e5205db643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:46.990109 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e54b8b-77b5-431f-9003-07e5205db643 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:46:47.045121 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:46:47.108518 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35ab419-e46f-4638-8cd7-243c36f0efd0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bfdb655a2a44b33ab724daaf73a294b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:46:47.109073 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ab419-e46f-4638-8cd7-243c36f0efd0 service nova] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:47.109361 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ab419-e46f-4638-8cd7-243c36f0efd0 service nova] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:47.109637 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ab419-e46f-4638-8cd7-243c36f0efd0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:47.111327 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-b35ab419-e46f-4638-8cd7-243c36f0efd0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:47.201698 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35ab419-e46f-4638-8cd7-243c36f0efd0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:46:47.203365 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35ab419-e46f-4638-8cd7-243c36f0efd0 service nova] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:46:47.222238 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 726/1437] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:46:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:46:48.070568 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b1f59-ad1c-469d-acf1-20238d9aa5c4 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:48.071119 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b1f59-ad1c-469d-acf1-20238d9aa5c4 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:48.071252 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b1f59-ad1c-469d-acf1-20238d9aa5c4 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:48.072569 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 712/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 22:46:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:46:48.077554 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b034d06-554c-4669-8825-94b54f592c73 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:48.077973 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b034d06-554c-4669-8825-94b54f592c73 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:48.078085 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b034d06-554c-4669-8825-94b54f592c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:48.154859 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-2b034d06-554c-4669-8825-94b54f592c73 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:46:48.156069 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-2b034d06-554c-4669-8825-94b54f592c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:48.158100 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:46:48.252906 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 727/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 22:46:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:46:48.404954 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472b1f59-ad1c-469d-acf1-20238d9aa5c4 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:46:48.406569 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-472b1f59-ad1c-469d-acf1-20238d9aa5c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:48.554697 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472b1f59-ad1c-469d-acf1-20238d9aa5c4 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:46:48.582642 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:46:48.668508 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34614f47-24d1-407e-885d-a4cae7456239 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bfdb655a2a44b33ab724daaf73a294b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:46:48.669116 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34614f47-24d1-407e-885d-a4cae7456239 service nova] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:48.669403 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34614f47-24d1-407e-885d-a4cae7456239 service nova] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:48.669681 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34614f47-24d1-407e-885d-a4cae7456239 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:48.671411 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-34614f47-24d1-407e-885d-a4cae7456239 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:48.763869 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34614f47-24d1-407e-885d-a4cae7456239 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:46:48.765454 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34614f47-24d1-407e-885d-a4cae7456239 service nova] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:46:48.783720 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 713/1440] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:46:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:46:50.717347 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df19c3-a56b-4769-8783-786d07e07f4b None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:50.717920 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df19c3-a56b-4769-8783-786d07e07f4b None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:50.717920 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df19c3-a56b-4769-8783-786d07e07f4b None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:50.719308 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 728/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 22:46:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:46:50.724466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08834a8b-fc27-4495-a267-43b90f93daf5 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:50.724888 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08834a8b-fc27-4495-a267-43b90f93daf5 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:50.725052 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08834a8b-fc27-4495-a267-43b90f93daf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:50.765040 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:46:50.790020 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-08834a8b-fc27-4495-a267-43b90f93daf5 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:46:50.791295 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-08834a8b-fc27-4495-a267-43b90f93daf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:50.892515 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 714/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 22:46:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:46:50.933627 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2df19c3-a56b-4769-8783-786d07e07f4b None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:46:50.935235 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-d2df19c3-a56b-4769-8783-786d07e07f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:51.038406 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2df19c3-a56b-4769-8783-786d07e07f4b None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:46:51.070302 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:46:51.111898 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65ae3ee-742a-4d66-8882-2d859b07fe69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:46:51.112543 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ae3ee-742a-4d66-8882-2d859b07fe69 service neutron] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:51.112827 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ae3ee-742a-4d66-8882-2d859b07fe69 service neutron] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:51.113102 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ae3ee-742a-4d66-8882-2d859b07fe69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:51.114875 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-b65ae3ee-742a-4d66-8882-2d859b07fe69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:51.226972 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65ae3ee-742a-4d66-8882-2d859b07fe69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:46:51.229061 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65ae3ee-742a-4d66-8882-2d859b07fe69 service neutron] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:46:51.246844 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 729/1443] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:46:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:46:53.427696 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a487c-50df-4606-b346-0a3b4d064fcc None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:53.428232 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a487c-50df-4606-b346-0a3b4d064fcc None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:53.428232 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a487c-50df-4606-b346-0a3b4d064fcc None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:53.429671 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 715/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 22:46:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:46:53.435558 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d973ad3-48ba-40ed-8ef6-dae2466aa6ce None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:53.435972 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d973ad3-48ba-40ed-8ef6-dae2466aa6ce None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:53.436130 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d973ad3-48ba-40ed-8ef6-dae2466aa6ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:53.501691 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-9d973ad3-48ba-40ed-8ef6-dae2466aa6ce None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:46:53.502969 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-9d973ad3-48ba-40ed-8ef6-dae2466aa6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:53.516335 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:46:53.603548 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 730/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 22:46:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:46:53.642447 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a487c-50df-4606-b346-0a3b4d064fcc None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:46:53.644116 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-601a487c-50df-4606-b346-0a3b4d064fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:53.747506 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a487c-50df-4606-b346-0a3b4d064fcc None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:46:53.759977 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:46:53.820325 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6c0d66-4045-4cbd-ba32-9cf0c0868fa6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:46:53.820976 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c0d66-4045-4cbd-ba32-9cf0c0868fa6 service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:53.821263 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c0d66-4045-4cbd-ba32-9cf0c0868fa6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:53.822034 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c0d66-4045-4cbd-ba32-9cf0c0868fa6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:53.823737 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-5b6c0d66-4045-4cbd-ba32-9cf0c0868fa6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:53.933050 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6c0d66-4045-4cbd-ba32-9cf0c0868fa6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:46:53.934483 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6c0d66-4045-4cbd-ba32-9cf0c0868fa6 service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:46:53.953872 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 716/1446] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:46:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:46:56.332581 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce89249-b4c9-45dc-9e6e-a94ded61f7a5 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:56.333140 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce89249-b4c9-45dc-9e6e-a94ded61f7a5 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:56.333140 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce89249-b4c9-45dc-9e6e-a94ded61f7a5 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:56.334533 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 731/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 22:46:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 22:46:56.339828 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213c363-e8dc-4e45-b50c-73b3f3c484d6 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:56.340251 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213c363-e8dc-4e45-b50c-73b3f3c484d6 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:56.340400 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213c363-e8dc-4e45-b50c-73b3f3c484d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:56.387654 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:46:56.405127 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-d213c363-e8dc-4e45-b50c-73b3f3c484d6 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:46:56.412297 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-d213c363-e8dc-4e45-b50c-73b3f3c484d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:56.504014 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 717/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 22:46:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:46:56.543479 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce89249-b4c9-45dc-9e6e-a94ded61f7a5 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:46:56.545025 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-cce89249-b4c9-45dc-9e6e-a94ded61f7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:56.647128 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce89249-b4c9-45dc-9e6e-a94ded61f7a5 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:46:56.686868 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:46:56.720881 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1252ce4-a987-4a39-aa61-90088208001b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:46:56.721537 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1252ce4-a987-4a39-aa61-90088208001b service neutron] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:56.721819 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1252ce4-a987-4a39-aa61-90088208001b service neutron] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:56.722096 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1252ce4-a987-4a39-aa61-90088208001b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:56.723827 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-a1252ce4-a987-4a39-aa61-90088208001b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:56.835836 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1252ce4-a987-4a39-aa61-90088208001b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:46:56.837952 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1252ce4-a987-4a39-aa61-90088208001b service neutron] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:46:56.855926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 732/1449] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:46:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:46:59.063870 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12de2639-00fb-495a-b72f-b6dfc61b56ad None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:59.064417 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12de2639-00fb-495a-b72f-b6dfc61b56ad None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:59.064417 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12de2639-00fb-495a-b72f-b6dfc61b56ad None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:59.065871 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 718/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 22:46:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:46:59.071692 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0d3f4-a3fc-4b46-a403-5ad7e6d1f09c None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:59.072140 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0d3f4-a3fc-4b46-a403-5ad7e6d1f09c None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:59.072287 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0d3f4-a3fc-4b46-a403-5ad7e6d1f09c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:59.136774 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-4ac0d3f4-a3fc-4b46-a403-5ad7e6d1f09c None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:46:59.137988 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-4ac0d3f4-a3fc-4b46-a403-5ad7e6d1f09c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:59.159174 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:46:59.241152 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 733/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 22:46:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:46:59.280697 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12de2639-00fb-495a-b72f-b6dfc61b56ad None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:46:59.282237 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-12de2639-00fb-495a-b72f-b6dfc61b56ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:59.386645 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12de2639-00fb-495a-b72f-b6dfc61b56ad None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:46:59.405961 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:46:59.460247 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed7c40b-4bfa-43c1-8402-f15ab62ddad4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:46:59.460885 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed7c40b-4bfa-43c1-8402-f15ab62ddad4 service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:46:59.461218 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed7c40b-4bfa-43c1-8402-f15ab62ddad4 service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:46:59.461493 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed7c40b-4bfa-43c1-8402-f15ab62ddad4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:46:59.463208 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-9ed7c40b-4bfa-43c1-8402-f15ab62ddad4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:46:59.577010 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed7c40b-4bfa-43c1-8402-f15ab62ddad4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:46:59.578434 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed7c40b-4bfa-43c1-8402-f15ab62ddad4 service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:46:59.597055 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 719/1452] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:46:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:47:01.824011 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3340830a-cd8e-44f6-a64a-7daccf62ec36 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:47:01.824514 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3340830a-cd8e-44f6-a64a-7daccf62ec36 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:47:01.824514 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3340830a-cd8e-44f6-a64a-7daccf62ec36 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:47:01.825866 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 734/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 22:47:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:47:01.831095 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58411f4f-1343-4cff-aa45-b76ac67004c5 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:47:01.831490 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58411f4f-1343-4cff-aa45-b76ac67004c5 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:47:01.831664 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58411f4f-1343-4cff-aa45-b76ac67004c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:47:01.893814 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:47:01.896734 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-58411f4f-1343-4cff-aa45-b76ac67004c5 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:47:01.897946 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-58411f4f-1343-4cff-aa45-b76ac67004c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:47:01.993905 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 720/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 22:47:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:47:02.033116 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3340830a-cd8e-44f6-a64a-7daccf62ec36 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:47:02.034678 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-3340830a-cd8e-44f6-a64a-7daccf62ec36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:47:02.145870 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3340830a-cd8e-44f6-a64a-7daccf62ec36 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:47:02.195586 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:47:02.220747 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf259601-8180-4cf3-8d3a-01f993ec95a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:47:02.221389 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf259601-8180-4cf3-8d3a-01f993ec95a6 service neutron] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:47:02.221673 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf259601-8180-4cf3-8d3a-01f993ec95a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:47:02.221950 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf259601-8180-4cf3-8d3a-01f993ec95a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:47:02.223624 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-bf259601-8180-4cf3-8d3a-01f993ec95a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:47:02.335344 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf259601-8180-4cf3-8d3a-01f993ec95a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:47:02.336803 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf259601-8180-4cf3-8d3a-01f993ec95a6 service neutron] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:47:02.355358 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 735/1455] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:47:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:47:04.451511 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ddc909-acab-4a4c-8b7d-224c03261ff1 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:47:04.452102 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ddc909-acab-4a4c-8b7d-224c03261ff1 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:47:04.452102 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ddc909-acab-4a4c-8b7d-224c03261ff1 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:47:04.453488 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 721/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 22:47:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:47:04.458661 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174bc8e-d681-42eb-bc85-2f0c45284384 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:47:04.459106 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174bc8e-d681-42eb-bc85-2f0c45284384 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:47:04.459229 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174bc8e-d681-42eb-bc85-2f0c45284384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:47:04.523600 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-6174bc8e-d681-42eb-bc85-2f0c45284384 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:47:04.524828 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-6174bc8e-d681-42eb-bc85-2f0c45284384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:47:04.553820 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:47:04.622509 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 736/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 22:47:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:47:04.660846 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ddc909-acab-4a4c-8b7d-224c03261ff1 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:47:04.662388 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-98ddc909-acab-4a4c-8b7d-224c03261ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:47:04.765387 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ddc909-acab-4a4c-8b7d-224c03261ff1 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:47:04.793424 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:47:04.839059 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a4bd48-1967-49dd-9c70-e941d3d9df8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:47:04.839707 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a4bd48-1967-49dd-9c70-e941d3d9df8e service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:47:04.839993 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a4bd48-1967-49dd-9c70-e941d3d9df8e service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:47:04.840272 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a4bd48-1967-49dd-9c70-e941d3d9df8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:47:04.841922 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-f8a4bd48-1967-49dd-9c70-e941d3d9df8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:47:04.950859 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a4bd48-1967-49dd-9c70-e941d3d9df8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:47:04.952274 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a4bd48-1967-49dd-9c70-e941d3d9df8e service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:47:04.971727 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 722/1458] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:47:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:47:07.060018 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b4ce20-6dcd-49ba-93d4-9907b715e3e0 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:47:07.060533 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b4ce20-6dcd-49ba-93d4-9907b715e3e0 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:47:07.060629 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b4ce20-6dcd-49ba-93d4-9907b715e3e0 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:47:07.061944 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 737/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 22:47:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:47:07.067198 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260b0fcb-5245-4ffc-9efe-ba9ae7fd4433 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:47:07.067659 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260b0fcb-5245-4ffc-9efe-ba9ae7fd4433 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:47:07.067771 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260b0fcb-5245-4ffc-9efe-ba9ae7fd4433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:47:07.131367 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-260b0fcb-5245-4ffc-9efe-ba9ae7fd4433 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:47:07.132575 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-260b0fcb-5245-4ffc-9efe-ba9ae7fd4433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:47:07.141246 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:47:07.236874 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 723/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 22:47:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:47:07.275438 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b4ce20-6dcd-49ba-93d4-9907b715e3e0 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:47:07.276963 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-25b4ce20-6dcd-49ba-93d4-9907b715e3e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:47:07.380703 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b4ce20-6dcd-49ba-93d4-9907b715e3e0 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:47:07.436911 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:47:07.455348 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7972ca-17a1-4ef7-8033-25ce92478e3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:47:07.455993 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7972ca-17a1-4ef7-8033-25ce92478e3d service neutron] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:47:07.456276 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7972ca-17a1-4ef7-8033-25ce92478e3d service neutron] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:47:07.456554 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7972ca-17a1-4ef7-8033-25ce92478e3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:47:07.458193 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-4b7972ca-17a1-4ef7-8033-25ce92478e3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:47:07.570537 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7972ca-17a1-4ef7-8033-25ce92478e3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:47:07.572011 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7972ca-17a1-4ef7-8033-25ce92478e3d service neutron] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:47:07.590364 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 738/1461] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:47:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:47:09.653918 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccabcd3-7e6f-4ab9-9ded-091b27e8fee4 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:47:09.654443 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccabcd3-7e6f-4ab9-9ded-091b27e8fee4 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:47:09.654565 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccabcd3-7e6f-4ab9-9ded-091b27e8fee4 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:47:09.655932 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 724/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 22:47:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:47:09.661366 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46488389-796f-4f1d-a826-28115194088f None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:47:09.661760 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46488389-796f-4f1d-a826-28115194088f None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:47:09.661926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46488389-796f-4f1d-a826-28115194088f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:47:09.725895 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-46488389-796f-4f1d-a826-28115194088f None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:47:09.727909 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-46488389-796f-4f1d-a826-28115194088f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:47:09.765117 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:47:09.823315 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 739/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 22:47:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:47:09.868566 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccabcd3-7e6f-4ab9-9ded-091b27e8fee4 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:47:09.870110 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-eccabcd3-7e6f-4ab9-9ded-091b27e8fee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:47:09.972140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccabcd3-7e6f-4ab9-9ded-091b27e8fee4 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:47:10.009540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:47:10.045620 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdfe409-038d-4bde-9f91-7aeb6976e3c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:47:10.046192 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdfe409-038d-4bde-9f91-7aeb6976e3c1 service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:47:10.046470 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdfe409-038d-4bde-9f91-7aeb6976e3c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:47:10.046741 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdfe409-038d-4bde-9f91-7aeb6976e3c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:47:10.048405 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-7fdfe409-038d-4bde-9f91-7aeb6976e3c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:47:10.155020 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdfe409-038d-4bde-9f91-7aeb6976e3c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:47:10.156464 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdfe409-038d-4bde-9f91-7aeb6976e3c1 service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:47:10.174696 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 725/1464] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:47:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:47:12.271649 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9559facc-5d83-4512-9401-4187a692a14e None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:47:12.272170 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9559facc-5d83-4512-9401-4187a692a14e None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:47:12.272170 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9559facc-5d83-4512-9401-4187a692a14e None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:47:12.273489 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 740/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 22:47:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:47:12.279297 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad42dc-ecca-4937-b734-3bc6c7de7c01 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:47:12.279693 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad42dc-ecca-4937-b734-3bc6c7de7c01 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:47:12.279852 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad42dc-ecca-4937-b734-3bc6c7de7c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:47:12.344743 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-5fad42dc-ecca-4937-b734-3bc6c7de7c01 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:47:12.345947 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-5fad42dc-ecca-4937-b734-3bc6c7de7c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:47:12.363838 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:47:12.442023 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 726/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 22:47:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:47:12.480966 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9559facc-5d83-4512-9401-4187a692a14e None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:47:12.482500 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-9559facc-5d83-4512-9401-4187a692a14e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:47:12.585155 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9559facc-5d83-4512-9401-4187a692a14e None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:47:12.651950 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:47:12.658371 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f95c69-32fd-4cdf-b086-8bf159e30561 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:47:12.659054 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f95c69-32fd-4cdf-b086-8bf159e30561 service neutron] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:47:12.659347 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f95c69-32fd-4cdf-b086-8bf159e30561 service neutron] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:47:12.659625 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f95c69-32fd-4cdf-b086-8bf159e30561 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:47:12.661277 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-c0f95c69-32fd-4cdf-b086-8bf159e30561 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:47:12.771337 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f95c69-32fd-4cdf-b086-8bf159e30561 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:47:12.772769 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f95c69-32fd-4cdf-b086-8bf159e30561 service neutron] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:47:12.790656 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 741/1467] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:47:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:47:14.846215 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02ac43-e6ba-45c3-bd9c-b27810adf0e5 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:47:14.846831 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02ac43-e6ba-45c3-bd9c-b27810adf0e5 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:47:14.846985 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02ac43-e6ba-45c3-bd9c-b27810adf0e5 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:47:14.848390 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 727/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 22:47:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:47:14.854323 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523e1e5-6b3c-4284-8696-c43f50f0576f None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:47:14.854856 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523e1e5-6b3c-4284-8696-c43f50f0576f None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:47:14.855351 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523e1e5-6b3c-4284-8696-c43f50f0576f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:47:14.921961 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-8523e1e5-6b3c-4284-8696-c43f50f0576f None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:47:14.923246 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-8523e1e5-6b3c-4284-8696-c43f50f0576f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:47:14.970857 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:47:15.019287 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 742/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 22:47:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:47:15.056892 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae02ac43-e6ba-45c3-bd9c-b27810adf0e5 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:47:15.058431 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-ae02ac43-e6ba-45c3-bd9c-b27810adf0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:47:15.162214 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae02ac43-e6ba-45c3-bd9c-b27810adf0e5 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:47:15.208758 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:47:15.236077 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cb5474-785a-439c-a177-4380ac425ba9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:47:15.236720 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb5474-785a-439c-a177-4380ac425ba9 service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:47:15.237003 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb5474-785a-439c-a177-4380ac425ba9 service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:47:15.237276 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb5474-785a-439c-a177-4380ac425ba9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:47:15.238964 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-30cb5474-785a-439c-a177-4380ac425ba9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:47:15.347902 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cb5474-785a-439c-a177-4380ac425ba9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:47:15.349328 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cb5474-785a-439c-a177-4380ac425ba9 service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:47:15.367825 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 728/1470] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:47:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:47:25.017420 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=22 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:47:25.365496 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=66 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:47:36.024967 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec8c405-872c-4d8b-873f-3fd9d1796736 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:47:36.025503 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec8c405-872c-4d8b-873f-3fd9d1796736 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:47:36.025503 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec8c405-872c-4d8b-873f-3fd9d1796736 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:47:36.026904 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 743/1471] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:47:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:47:36.032133 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3304a4-edf7-4a5a-88e6-4df0aacd7c63 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:47:36.032588 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3304a4-edf7-4a5a-88e6-4df0aacd7c63 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:47:36.032698 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3304a4-edf7-4a5a-88e6-4df0aacd7c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:47:36.097465 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-1f3304a4-edf7-4a5a-88e6-4df0aacd7c63 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:47:36.098675 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-1f3304a4-edf7-4a5a-88e6-4df0aacd7c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:47:36.197703 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 729/1472] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:47:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:47:36.282182 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c14afc-e054-42a6-a633-4b4fa48c97b7 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:47:36.282477 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c14afc-e054-42a6-a633-4b4fa48c97b7 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:47:36.282762 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c14afc-e054-42a6-a633-4b4fa48c97b7 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:47:36.284145 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 744/1473] 10.222.0.52 () {54 vars in 966 bytes} [Thu Jul 21 22:47:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:47:36.289676 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c2e9e-c977-4133-8c6f-49101c64860e None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:47:36.290010 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c2e9e-c977-4133-8c6f-49101c64860e None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:47:36.290287 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c2e9e-c977-4133-8c6f-49101c64860e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:47:36.363247 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-ed5c2e9e-c977-4133-8c6f-49101c64860e None None] MFA Rules not processed for user `927778d005fd48a4b4b8fd3c4419bb36`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:47:36.364450 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-ed5c2e9e-c977-4133-8c6f-49101c64860e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:47:36.384731 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:47:36.438720 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 730/1474] 10.222.0.52 () {58 vars in 1063 bytes} [Thu Jul 21 22:47:36 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:47:36.446818 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c14afc-e054-42a6-a633-4b4fa48c97b7 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:47:36.448335 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-59c14afc-e054-42a6-a633-4b4fa48c97b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:47:36.551102 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c14afc-e054-42a6-a633-4b4fa48c97b7 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:47:36.624868 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e00f6-dac3-41af-bfe3-f5156a93b003 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '927778d005fd48a4b4b8fd3c4419bb36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:47:36.625513 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e00f6-dac3-41af-bfe3-f5156a93b003 service glance] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:47:36.625796 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e00f6-dac3-41af-bfe3-f5156a93b003 service glance] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:47:36.626069 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e00f6-dac3-41af-bfe3-f5156a93b003 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:47:36.627752 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-f45e00f6-dac3-41af-bfe3-f5156a93b003 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:47:36.684536 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:47:36.738558 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45e00f6-dac3-41af-bfe3-f5156a93b003 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:47:36.740017 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45e00f6-dac3-41af-bfe3-f5156a93b003 service glance] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:47:36.760304 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 745/1475] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:47:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:47:38.862583 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c8c0d-f7d0-4f7f-8407-68bba2eafba7 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:47:38.863165 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c8c0d-f7d0-4f7f-8407-68bba2eafba7 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:47:38.863165 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c8c0d-f7d0-4f7f-8407-68bba2eafba7 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:47:38.864569 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 731/1476] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:47:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:47:38.869726 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb05fa6c-47d7-479b-b53d-32b47a9c5f19 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:47:38.870161 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb05fa6c-47d7-479b-b53d-32b47a9c5f19 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:47:38.870319 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb05fa6c-47d7-479b-b53d-32b47a9c5f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:47:38.934569 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-eb05fa6c-47d7-479b-b53d-32b47a9c5f19 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:47:38.935811 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-eb05fa6c-47d7-479b-b53d-32b47a9c5f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:47:39.033697 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 746/1477] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:47:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:47:39.057411 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68c8c0d-f7d0-4f7f-8407-68bba2eafba7 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:47:39.058983 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-a68c8c0d-f7d0-4f7f-8407-68bba2eafba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:47:39.183148 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68c8c0d-f7d0-4f7f-8407-68bba2eafba7 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:47:39.235399 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:47:39.279549 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4e7b42-f14a-44fc-9291-d89767fbb4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:47:39.280205 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e7b42-f14a-44fc-9291-d89767fbb4a9 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:47:39.280488 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e7b42-f14a-44fc-9291-d89767fbb4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:47:39.280765 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e7b42-f14a-44fc-9291-d89767fbb4a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:47:39.281494 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4e7b42-f14a-44fc-9291-d89767fbb4a9 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:47:39.283240 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: WARNING py.warnings [None req-2b4e7b42-f14a-44fc-9291-d89767fbb4a9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 22:47:39.283240 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: warnings.warn(msg) Jul 21 22:47:39.283240 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: Jul 21 22:47:39.284374 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4e7b42-f14a-44fc-9291-d89767fbb4a9 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:47:39.304459 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 732/1478] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Jul 21 22:47:39 2022] GET /identity/v3/projects => generated 2553 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:47:41.352175 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93179934-4fbd-42ac-8a73-09ed9e821bb5 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:47:41.352709 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93179934-4fbd-42ac-8a73-09ed9e821bb5 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:47:41.352709 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93179934-4fbd-42ac-8a73-09ed9e821bb5 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:47:41.354098 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 747/1479] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:47:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:47:41.359767 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60b0b9-81cf-40fa-85f5-647a602a2f81 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:47:41.360186 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60b0b9-81cf-40fa-85f5-647a602a2f81 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:47:41.360350 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60b0b9-81cf-40fa-85f5-647a602a2f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:47:41.424925 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-0d60b0b9-81cf-40fa-85f5-647a602a2f81 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:47:41.426223 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-0d60b0b9-81cf-40fa-85f5-647a602a2f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:47:41.525617 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 733/1480] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:47:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:47:41.569314 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93179934-4fbd-42ac-8a73-09ed9e821bb5 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:47:41.570924 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-93179934-4fbd-42ac-8a73-09ed9e821bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:47:41.580892 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:47:41.717473 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93179934-4fbd-42ac-8a73-09ed9e821bb5 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:47:41.796719 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:47:41.829944 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968cd831-ce36-4b12-884d-0cc10865e2bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bfdb655a2a44b33ab724daaf73a294b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:47:41.830589 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968cd831-ce36-4b12-884d-0cc10865e2bf service nova] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:47:41.830930 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968cd831-ce36-4b12-884d-0cc10865e2bf service nova] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:47:41.831215 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968cd831-ce36-4b12-884d-0cc10865e2bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:47:41.832850 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-968cd831-ce36-4b12-884d-0cc10865e2bf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:47:41.923084 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968cd831-ce36-4b12-884d-0cc10865e2bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:47:41.924790 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968cd831-ce36-4b12-884d-0cc10865e2bf service nova] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:47:41.945690 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 748/1481] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:47:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:47:44.183717 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb82c0d-533e-4fc3-9a1e-7a3c1dbc6b93 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:47:44.184162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb82c0d-533e-4fc3-9a1e-7a3c1dbc6b93 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:47:44.184309 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb82c0d-533e-4fc3-9a1e-7a3c1dbc6b93 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:47:44.185745 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 734/1482] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:47:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:47:44.191207 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94104e20-db47-48a6-80ef-261b2c57bb62 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:47:44.191639 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94104e20-db47-48a6-80ef-261b2c57bb62 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:47:44.191806 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94104e20-db47-48a6-80ef-261b2c57bb62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:47:44.258477 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-94104e20-db47-48a6-80ef-261b2c57bb62 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:47:44.259715 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-94104e20-db47-48a6-80ef-261b2c57bb62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:47:44.307876 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:47:44.357615 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 749/1483] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:47:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:47:44.394982 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb82c0d-533e-4fc3-9a1e-7a3c1dbc6b93 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:47:44.396535 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-ceb82c0d-533e-4fc3-9a1e-7a3c1dbc6b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:47:44.419696 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:47:44.501098 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb82c0d-533e-4fc3-9a1e-7a3c1dbc6b93 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:47:44.576767 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fce1fa9-521e-4d54-82e6-471a8c872d55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:47:44.577448 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce1fa9-521e-4d54-82e6-471a8c872d55 service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:47:44.577762 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce1fa9-521e-4d54-82e6-471a8c872d55 service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:47:44.578039 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce1fa9-521e-4d54-82e6-471a8c872d55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:47:44.579738 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-2fce1fa9-521e-4d54-82e6-471a8c872d55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:47:44.654173 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:47:44.689086 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fce1fa9-521e-4d54-82e6-471a8c872d55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:47:44.690554 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fce1fa9-521e-4d54-82e6-471a8c872d55 service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:47:44.709307 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 735/1484] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:47:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:47:46.804696 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc5b8a7-3725-4158-9387-0ebaa82d1080 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:47:46.805139 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc5b8a7-3725-4158-9387-0ebaa82d1080 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:47:46.805295 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc5b8a7-3725-4158-9387-0ebaa82d1080 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:47:46.806601 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 750/1485] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:47:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:47:46.811911 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467665b3-af97-4b0b-b87c-b29939b89f46 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:47:46.812322 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467665b3-af97-4b0b-b87c-b29939b89f46 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:47:46.812457 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467665b3-af97-4b0b-b87c-b29939b89f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:47:46.878514 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-467665b3-af97-4b0b-b87c-b29939b89f46 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:47:46.879769 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-467665b3-af97-4b0b-b87c-b29939b89f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:47:46.977954 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 736/1486] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:47:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:47:47.016808 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc5b8a7-3725-4158-9387-0ebaa82d1080 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:47:47.018395 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-1dc5b8a7-3725-4158-9387-0ebaa82d1080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:47:47.122212 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc5b8a7-3725-4158-9387-0ebaa82d1080 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:47:47.194190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:47:47.196026 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecbfcdd-bc13-4659-ad2d-a3a9dccbf3fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:47:47.196677 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecbfcdd-bc13-4659-ad2d-a3a9dccbf3fd service neutron] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:47:47.196965 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecbfcdd-bc13-4659-ad2d-a3a9dccbf3fd service neutron] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:47:47.197242 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecbfcdd-bc13-4659-ad2d-a3a9dccbf3fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:47:47.198937 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-eecbfcdd-bc13-4659-ad2d-a3a9dccbf3fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:47:47.309924 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecbfcdd-bc13-4659-ad2d-a3a9dccbf3fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:47:47.311398 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecbfcdd-bc13-4659-ad2d-a3a9dccbf3fd service neutron] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:47:47.329126 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 751/1487] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:47:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:47:49.531319 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e61d9-9066-4d3b-aa72-c614948f4edb None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:47:49.531852 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e61d9-9066-4d3b-aa72-c614948f4edb None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:47:49.531852 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e61d9-9066-4d3b-aa72-c614948f4edb None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:47:49.533253 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 737/1488] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:47:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:47:49.539889 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53411f5-aadb-4d8e-bdd7-26849d16866b None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:47:49.540323 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53411f5-aadb-4d8e-bdd7-26849d16866b None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:47:49.540429 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53411f5-aadb-4d8e-bdd7-26849d16866b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:47:49.604244 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-b53411f5-aadb-4d8e-bdd7-26849d16866b None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:47:49.605439 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-b53411f5-aadb-4d8e-bdd7-26849d16866b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:47:49.657368 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:47:49.700482 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 752/1489] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:47:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:47:49.741960 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2e61d9-9066-4d3b-aa72-c614948f4edb None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:47:49.743548 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-cc2e61d9-9066-4d3b-aa72-c614948f4edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:47:49.781476 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:47:49.848175 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2e61d9-9066-4d3b-aa72-c614948f4edb None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:47:49.922464 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b1e059-4ca7-4781-b483-4401a9acb998 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:47:49.923139 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b1e059-4ca7-4781-b483-4401a9acb998 service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:47:49.923428 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b1e059-4ca7-4781-b483-4401a9acb998 service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:47:49.923706 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b1e059-4ca7-4781-b483-4401a9acb998 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:47:49.925394 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-b0b1e059-4ca7-4781-b483-4401a9acb998 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:47:50.004291 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:47:50.033436 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b1e059-4ca7-4781-b483-4401a9acb998 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:47:50.034814 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b1e059-4ca7-4781-b483-4401a9acb998 service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:47:50.053505 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 738/1490] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:47:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:47:52.468198 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a7f4c-42af-43f7-ac08-8557992cf640 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:47:52.468786 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a7f4c-42af-43f7-ac08-8557992cf640 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:47:52.468786 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a7f4c-42af-43f7-ac08-8557992cf640 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:47:52.470057 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 753/1491] 10.222.0.52 () {54 vars in 926 bytes} [Thu Jul 21 22:47:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:47:52.475703 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a742e9c-552f-4de1-b430-8e19fa1036c8 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:47:52.476136 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a742e9c-552f-4de1-b430-8e19fa1036c8 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:47:52.476408 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a742e9c-552f-4de1-b430-8e19fa1036c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:47:52.543652 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-4a742e9c-552f-4de1-b430-8e19fa1036c8 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:47:52.544874 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-4a742e9c-552f-4de1-b430-8e19fa1036c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:47:52.639783 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 739/1492] 10.222.0.52 () {58 vars in 1023 bytes} [Thu Jul 21 22:47:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:47:52.679383 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136a7f4c-42af-43f7-ac08-8557992cf640 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:47:52.680908 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-136a7f4c-42af-43f7-ac08-8557992cf640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:47:52.787666 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136a7f4c-42af-43f7-ac08-8557992cf640 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:47:52.866342 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317e0fa-4af0-4b23-9268-023fa91dec8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:47:52.867024 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317e0fa-4af0-4b23-9268-023fa91dec8d service neutron] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:47:52.867311 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317e0fa-4af0-4b23-9268-023fa91dec8d service neutron] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:47:52.867589 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317e0fa-4af0-4b23-9268-023fa91dec8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:47:52.869258 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-5317e0fa-4af0-4b23-9268-023fa91dec8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:47:52.878991 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:47:52.983085 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5317e0fa-4af0-4b23-9268-023fa91dec8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:47:52.984512 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5317e0fa-4af0-4b23-9268-023fa91dec8d service neutron] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:47:53.002031 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 754/1493] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:47:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:48:02.637846 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=82 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:48:02.999780 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=50 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:49:27.972316 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeab688-085f-4209-acd0-cc07a9974854 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:49:27.972924 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeab688-085f-4209-acd0-cc07a9974854 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:49:27.972924 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeab688-085f-4209-acd0-cc07a9974854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:49:28.047313 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-adeab688-085f-4209-acd0-cc07a9974854 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:49:28.048525 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-adeab688-085f-4209-acd0-cc07a9974854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:49:28.149018 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 740/1494] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:49:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:49:28.156996 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eacd91-2819-461e-9833-f6feac50d61f None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:49:28.157521 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eacd91-2819-461e-9833-f6feac50d61f None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:49:28.157629 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eacd91-2819-461e-9833-f6feac50d61f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:49:28.232131 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-c4eacd91-2819-461e-9833-f6feac50d61f None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:49:28.233282 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-c4eacd91-2819-461e-9833-f6feac50d61f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:49:28.334728 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 755/1495] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:49:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:49:28.339430 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeab688-085f-4209-acd0-cc07a9974854 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:49:28.340644 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-adeab688-085f-4209-acd0-cc07a9974854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:49:28.352343 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeab688-085f-4209-acd0-cc07a9974854 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:49:28.413218 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:49:28.447278 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e5b05b-4573-4dc5-83ea-767f6f3c150e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:49:28.447938 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5b05b-4573-4dc5-83ea-767f6f3c150e admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:49:28.448218 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5b05b-4573-4dc5-83ea-767f6f3c150e admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:49:28.448491 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5b05b-4573-4dc5-83ea-767f6f3c150e admin admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:49:28.449203 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e5b05b-4573-4dc5-83ea-767f6f3c150e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:49:28.449509 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e5b05b-4573-4dc5-83ea-767f6f3c150e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:49:28.450509 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: WARNING py.warnings [None req-09e5b05b-4573-4dc5-83ea-767f6f3c150e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 21 22:49:28.450509 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: warnings.warn(msg) Jul 21 22:49:28.450509 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: Jul 21 22:49:28.451418 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e5b05b-4573-4dc5-83ea-767f6f3c150e admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:49:28.462499 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 741/1496] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 22:49:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:49:28.470667 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eacd91-2819-461e-9833-f6feac50d61f None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:49:28.482238 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eacd91-2819-461e-9833-f6feac50d61f None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:49:28.660918 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:49:28.694305 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41fa7e9-0b70-45c4-aac7-be2e90a1bddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:49:28.695006 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41fa7e9-0b70-45c4-aac7-be2e90a1bddf admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:49:28.695291 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41fa7e9-0b70-45c4-aac7-be2e90a1bddf admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:49:28.695565 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41fa7e9-0b70-45c4-aac7-be2e90a1bddf admin admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:49:28.696252 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41fa7e9-0b70-45c4-aac7-be2e90a1bddf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:49:28.697972 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: WARNING py.warnings [None req-f41fa7e9-0b70-45c4-aac7-be2e90a1bddf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 22:49:28.697972 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: warnings.warn(msg) Jul 21 22:49:28.697972 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: Jul 21 22:49:28.699194 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41fa7e9-0b70-45c4-aac7-be2e90a1bddf admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:49:28.815368 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 756/1497] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:49:28 2022] POST /identity/v3/projects => generated 379 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:49:28.820281 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e5b05b-4573-4dc5-83ea-767f6f3c150e admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:49:28.821840 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-09e5b05b-4573-4dc5-83ea-767f6f3c150e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:49:28.857544 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e5b05b-4573-4dc5-83ea-767f6f3c150e admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:49:28.951487 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eab7d1-33ae-44e3-a029-65f637260cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:49:28.952150 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eab7d1-33ae-44e3-a029-65f637260cd7 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:49:28.952450 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eab7d1-33ae-44e3-a029-65f637260cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:49:28.952711 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eab7d1-33ae-44e3-a029-65f637260cd7 admin admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:49:28.953389 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eab7d1-33ae-44e3-a029-65f637260cd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:49:28.954966 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: WARNING py.warnings [None req-83eab7d1-33ae-44e3-a029-65f637260cd7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 22:49:28.954966 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: warnings.warn(msg) Jul 21 22:49:28.954966 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: Jul 21 22:49:28.956163 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eab7d1-33ae-44e3-a029-65f637260cd7 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:49:29.052261 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 742/1498] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:49:28 2022] POST /identity/v3/users => generated 290 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:49:29.056317 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41fa7e9-0b70-45c4-aac7-be2e90a1bddf admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:49:29.057809 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-f41fa7e9-0b70-45c4-aac7-be2e90a1bddf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:49:29.119075 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41fa7e9-0b70-45c4-aac7-be2e90a1bddf admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:49:29.201888 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:49:29.212008 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b79f7a-114e-4c38-905e-dbfc02383bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:49:29.212893 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b79f7a-114e-4c38-905e-dbfc02383bef admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:49:29.213005 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b79f7a-114e-4c38-905e-dbfc02383bef admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:49:29.213206 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b79f7a-114e-4c38-905e-dbfc02383bef admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:49:29.213931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b79f7a-114e-4c38-905e-dbfc02383bef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:49:29.214959 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: WARNING py.warnings [None req-06b79f7a-114e-4c38-905e-dbfc02383bef admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 21 22:49:29.214959 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: warnings.warn(msg) Jul 21 22:49:29.214959 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: Jul 21 22:49:29.215838 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b79f7a-114e-4c38-905e-dbfc02383bef admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:49:29.229143 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 757/1499] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:49:29 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:49:29.233147 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eab7d1-33ae-44e3-a029-65f637260cd7 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:49:29.235220 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-83eab7d1-33ae-44e3-a029-65f637260cd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:49:29.246704 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eab7d1-33ae-44e3-a029-65f637260cd7 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:49:29.272222 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:49:29.343791 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024431f0-f591-4e97-9033-ba605c312ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:49:29.344464 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024431f0-f591-4e97-9033-ba605c312ceb admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:49:29.344787 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024431f0-f591-4e97-9033-ba605c312ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:49:29.345062 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024431f0-f591-4e97-9033-ba605c312ceb admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:49:29.345860 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024431f0-f591-4e97-9033-ba605c312ceb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:49:29.346837 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: WARNING py.warnings [None req-024431f0-f591-4e97-9033-ba605c312ceb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 21 22:49:29.346837 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: warnings.warn(msg) Jul 21 22:49:29.346837 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: Jul 21 22:49:29.347728 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024431f0-f591-4e97-9033-ba605c312ceb admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:49:29.360859 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 743/1500] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:49:29 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:49:29.365103 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b79f7a-114e-4c38-905e-dbfc02383bef admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:49:29.366689 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-06b79f7a-114e-4c38-905e-dbfc02383bef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:49:29.400850 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b79f7a-114e-4c38-905e-dbfc02383bef admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:49:29.494058 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f37e4e-a6bd-4e0d-bdfe-2873b14638a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:49:29.494973 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f37e4e-a6bd-4e0d-bdfe-2873b14638a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:49:29.495258 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f37e4e-a6bd-4e0d-bdfe-2873b14638a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:49:29.495535 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f37e4e-a6bd-4e0d-bdfe-2873b14638a3 admin admin] PATH_INFO: `/v3/projects/39d26d85605c47db861fb84a090b0291/users/0c0d4918da334468821a972bda535ded/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:49:29.519565 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f37e4e-a6bd-4e0d-bdfe-2873b14638a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39d26d85605c47db861fb84a090b0291, user_id=0c0d4918da334468821a972bda535ded, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:49:29.520581 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: WARNING py.warnings [None req-79f37e4e-a6bd-4e0d-bdfe-2873b14638a3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 22:49:29.520581 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: warnings.warn(msg) Jul 21 22:49:29.520581 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: Jul 21 22:49:29.522479 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f37e4e-a6bd-4e0d-bdfe-2873b14638a3 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:49:29.595506 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 758/1501] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:49:29 2022] PUT /identity/v3/projects/39d26d85605c47db861fb84a090b0291/users/0c0d4918da334468821a972bda535ded/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:49:29.607908 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db437ef-be72-4645-b282-19c99ffd1dba None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:49:29.608246 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db437ef-be72-4645-b282-19c99ffd1dba None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:49:29.608523 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db437ef-be72-4645-b282-19c99ffd1dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:49:29.664627 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-3db437ef-be72-4645-b282-19c99ffd1dba None None] MFA Rules not processed for user `0c0d4918da334468821a972bda535ded`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:49:29.665778 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-3db437ef-be72-4645-b282-19c99ffd1dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:49:29.743344 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 744/1502] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:49:29 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:49:29.756955 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f7c61a-a5e0-4c62-a238-2618637ca0a5 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:49:29.757246 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f7c61a-a5e0-4c62-a238-2618637ca0a5 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:49:29.757526 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f7c61a-a5e0-4c62-a238-2618637ca0a5 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:49:29.758912 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 759/1503] 10.222.0.52 () {54 vars in 964 bytes} [Thu Jul 21 22:49:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:49:29.763872 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54d6c4-d691-47d2-a819-8f95e386d4be None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:49:29.764167 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54d6c4-d691-47d2-a819-8f95e386d4be None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:49:29.764458 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54d6c4-d691-47d2-a819-8f95e386d4be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:49:29.794686 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:49:29.839847 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-7d54d6c4-d691-47d2-a819-8f95e386d4be None None] MFA Rules not processed for user `150f42651f754fda83f25fe3b408bce9`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:49:29.841059 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-7d54d6c4-d691-47d2-a819-8f95e386d4be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:49:29.916681 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 745/1504] 10.222.0.52 () {58 vars in 1061 bytes} [Thu Jul 21 22:49:29 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:49:29.926296 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f7c61a-a5e0-4c62-a238-2618637ca0a5 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:49:29.930588 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-19f7c61a-a5e0-4c62-a238-2618637ca0a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:49:29.982712 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:49:30.039137 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f7c61a-a5e0-4c62-a238-2618637ca0a5 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:49:30.113285 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848347d4-2cd3-4388-8a43-e9a44c3aa20a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:49:30.113938 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848347d4-2cd3-4388-8a43-e9a44c3aa20a service cinder] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:49:30.114219 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848347d4-2cd3-4388-8a43-e9a44c3aa20a service cinder] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:49:30.114493 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848347d4-2cd3-4388-8a43-e9a44c3aa20a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:49:30.116183 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-848347d4-2cd3-4388-8a43-e9a44c3aa20a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:49:30.223237 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848347d4-2cd3-4388-8a43-e9a44c3aa20a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:49:30.224957 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848347d4-2cd3-4388-8a43-e9a44c3aa20a service cinder] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:49:30.228520 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:49:30.242487 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 760/1505] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:49:29 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:49:30.285333 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b889f5a-38e4-4965-b68c-5e0fbb89f74d None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:49:30.285755 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b889f5a-38e4-4965-b68c-5e0fbb89f74d None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:49:30.285917 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b889f5a-38e4-4965-b68c-5e0fbb89f74d None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:49:30.287248 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 746/1506] 10.222.0.52 () {52 vars in 829 bytes} [Thu Jul 21 22:49:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:49:30.290697 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848347d4-2cd3-4388-8a43-e9a44c3aa20a service cinder] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:49:30.291929 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-848347d4-2cd3-4388-8a43-e9a44c3aa20a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:49:30.392070 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848347d4-2cd3-4388-8a43-e9a44c3aa20a service cinder] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:49:30.484603 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454e8de6-5a24-4306-a732-4082147506d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:49:30.485495 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e8de6-5a24-4306-a732-4082147506d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:49:30.485776 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e8de6-5a24-4306-a732-4082147506d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:49:30.487560 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e8de6-5a24-4306-a732-4082147506d9 admin admin] PATH_INFO: `/v3/users/0c0d4918da334468821a972bda535ded` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:49:30.499500 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454e8de6-5a24-4306-a732-4082147506d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c0d4918da334468821a972bda535ded)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:49:30.501046 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: WARNING py.warnings [None req-454e8de6-5a24-4306-a732-4082147506d9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 22:49:30.501046 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: warnings.warn(msg) Jul 21 22:49:30.501046 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: Jul 21 22:49:30.502261 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454e8de6-5a24-4306-a732-4082147506d9 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:49:30.512726 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:49:30.720187 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-454e8de6-5a24-4306-a732-4082147506d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c0d4918da334468821a972bda535ded'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:49:30.732623 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-454e8de6-5a24-4306-a732-4082147506d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c0d4918da334468821a972bda535ded'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:49:30.733167 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-454e8de6-5a24-4306-a732-4082147506d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c0d4918da334468821a972bda535ded'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:49:30.751853 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-454e8de6-5a24-4306-a732-4082147506d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c0d4918da334468821a972bda535ded'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:49:30.752234 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-454e8de6-5a24-4306-a732-4082147506d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c0d4918da334468821a972bda535ded'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:49:30.800323 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 761/1507] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:49:30 2022] DELETE /identity/v3/users/0c0d4918da334468821a972bda535ded => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:49:30.804714 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f237e19d-f8ac-4cd8-bf1d-b021086409c4 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:49:30.805014 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f237e19d-f8ac-4cd8-bf1d-b021086409c4 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:49:30.805290 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f237e19d-f8ac-4cd8-bf1d-b021086409c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:49:30.860252 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-f237e19d-f8ac-4cd8-bf1d-b021086409c4 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:49:30.861474 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-f237e19d-f8ac-4cd8-bf1d-b021086409c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:49:30.934751 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:49:30.947802 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 747/1508] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:49:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:49:30.963513 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454e8de6-5a24-4306-a732-4082147506d9 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:49:30.964731 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-454e8de6-5a24-4306-a732-4082147506d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:49:31.065024 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454e8de6-5a24-4306-a732-4082147506d9 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:49:31.118693 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:49:31.137477 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcaf9b0-7d4f-4395-a6c1-38393292dc50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:49:31.138127 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcaf9b0-7d4f-4395-a6c1-38393292dc50 service neutron] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:49:31.138413 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcaf9b0-7d4f-4395-a6c1-38393292dc50 service neutron] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:49:31.138690 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcaf9b0-7d4f-4395-a6c1-38393292dc50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:49:31.140386 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-4dcaf9b0-7d4f-4395-a6c1-38393292dc50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:49:31.248989 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcaf9b0-7d4f-4395-a6c1-38393292dc50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:49:31.250966 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcaf9b0-7d4f-4395-a6c1-38393292dc50 service neutron] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:49:31.272376 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 762/1509] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:49:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:49:31.711026 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f237e19d-f8ac-4cd8-bf1d-b021086409c4 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:49:31.712336 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-f237e19d-f8ac-4cd8-bf1d-b021086409c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:49:31.725248 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f237e19d-f8ac-4cd8-bf1d-b021086409c4 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:49:31.826954 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22413f4b-9083-4296-bc75-04c16ec5781f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:49:31.827752 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22413f4b-9083-4296-bc75-04c16ec5781f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:49:31.828036 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22413f4b-9083-4296-bc75-04c16ec5781f admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:49:31.828972 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22413f4b-9083-4296-bc75-04c16ec5781f admin admin] PATH_INFO: `/v3/projects/39d26d85605c47db861fb84a090b0291` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:49:31.838950 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22413f4b-9083-4296-bc75-04c16ec5781f admin admin] RBAC: Authorizing `identity:delete_project(project_id=39d26d85605c47db861fb84a090b0291)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:49:31.839980 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: WARNING py.warnings [None req-22413f4b-9083-4296-bc75-04c16ec5781f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 22:49:31.839980 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: warnings.warn(msg) Jul 21 22:49:31.839980 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: Jul 21 22:49:31.841172 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22413f4b-9083-4296-bc75-04c16ec5781f admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:49:31.929402 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-22413f4b-9083-4296-bc75-04c16ec5781f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39d26d85605c47db861fb84a090b0291', 'request_id': 'req-22413f4b-9083-4296-bc75-04c16ec5781f'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:49:31.935409 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-22413f4b-9083-4296-bc75-04c16ec5781f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39d26d85605c47db861fb84a090b0291', 'request_id': 'req-22413f4b-9083-4296-bc75-04c16ec5781f'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:49:32.017735 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-22413f4b-9083-4296-bc75-04c16ec5781f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39d26d85605c47db861fb84a090b0291', 'request_id': 'req-22413f4b-9083-4296-bc75-04c16ec5781f'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:49:32.047036 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-22413f4b-9083-4296-bc75-04c16ec5781f admin admin] The token cache is being invalidate because project 39d26d85605c47db861fb84a090b0291 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:49:32.047387 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-22413f4b-9083-4296-bc75-04c16ec5781f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:49:32.047908 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-22413f4b-9083-4296-bc75-04c16ec5781f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:49:32.049272 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 748/1510] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:49:31 2022] DELETE /identity/v3/projects/39d26d85605c47db861fb84a090b0291 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:49:41.270094 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=62 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:49:42.045041 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:51:09.693510 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e43d5-7046-474c-8794-7780220d1af2 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:51:09.694110 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e43d5-7046-474c-8794-7780220d1af2 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:51:09.694110 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e43d5-7046-474c-8794-7780220d1af2 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:51:09.695496 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 763/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 22:51:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:51:09.700271 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c96feb4-2fcd-47c5-a801-ecd1886fb78c None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:51:09.700705 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c96feb4-2fcd-47c5-a801-ecd1886fb78c None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:51:09.700833 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c96feb4-2fcd-47c5-a801-ecd1886fb78c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:51:09.781880 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-5c96feb4-2fcd-47c5-a801-ecd1886fb78c None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:51:09.783030 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-5c96feb4-2fcd-47c5-a801-ecd1886fb78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:51:09.887064 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 749/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 22:51:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:51:10.041532 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01e43d5-7046-474c-8794-7780220d1af2 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:51:10.043632 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-f01e43d5-7046-474c-8794-7780220d1af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:51:10.160605 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01e43d5-7046-474c-8794-7780220d1af2 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:51:10.166004 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bbd81d-9238-45ff-be8b-f096c7c1b534 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bfdb655a2a44b33ab724daaf73a294b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:51:10.166642 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bbd81d-9238-45ff-be8b-f096c7c1b534 service nova] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:51:10.166957 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bbd81d-9238-45ff-be8b-f096c7c1b534 service nova] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:51:10.167239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bbd81d-9238-45ff-be8b-f096c7c1b534 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:51:10.175925 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bbd81d-9238-45ff-be8b-f096c7c1b534 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:51:10.177499 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bbd81d-9238-45ff-be8b-f096c7c1b534 service nova] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:51:10.186996 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 764/1513] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:51:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:51:11.018155 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc33a4-ee30-42ea-8442-d60cd4868f09 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:51:11.018718 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc33a4-ee30-42ea-8442-d60cd4868f09 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:51:11.018718 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc33a4-ee30-42ea-8442-d60cd4868f09 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:51:11.020109 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 750/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 22:51:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:51:11.024837 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc242c-b31e-4ba6-8a3a-63f859575eff None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:51:11.025258 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc242c-b31e-4ba6-8a3a-63f859575eff None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:51:11.025403 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc242c-b31e-4ba6-8a3a-63f859575eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:51:11.043855 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-1ddc242c-b31e-4ba6-8a3a-63f859575eff None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:51:11.045041 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-1ddc242c-b31e-4ba6-8a3a-63f859575eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:51:11.053527 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 765/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 22:51:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:51:11.202913 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dc33a4-ee30-42ea-8442-d60cd4868f09 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:51:11.216936 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dc33a4-ee30-42ea-8442-d60cd4868f09 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:51:11.223169 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544429aa-4215-45f3-a6fe-cb344df8c36a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bfdb655a2a44b33ab724daaf73a294b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:51:11.223797 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544429aa-4215-45f3-a6fe-cb344df8c36a service nova] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:51:11.224080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544429aa-4215-45f3-a6fe-cb344df8c36a service nova] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:51:11.224352 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544429aa-4215-45f3-a6fe-cb344df8c36a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:51:11.232336 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544429aa-4215-45f3-a6fe-cb344df8c36a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:51:11.233899 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544429aa-4215-45f3-a6fe-cb344df8c36a service nova] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:51:11.243603 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 751/1516] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:51:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:51:11.895349 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6956f1f-f322-47b5-a4f9-5469f4dbfc69 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:51:11.895643 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6956f1f-f322-47b5-a4f9-5469f4dbfc69 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:51:11.895923 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6956f1f-f322-47b5-a4f9-5469f4dbfc69 None None] PATH_INFO: `/v3` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:51:11.897226 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 766/1517] 10.222.0.33 () {54 vars in 925 bytes} [Thu Jul 21 22:51:11 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:51:11.903188 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f5aa3-4128-4b08-8f69-6db3a1489ebe None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:51:11.903480 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f5aa3-4128-4b08-8f69-6db3a1489ebe None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:51:11.903762 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f5aa3-4128-4b08-8f69-6db3a1489ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:51:11.963034 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-b10f5aa3-4128-4b08-8f69-6db3a1489ebe None None] MFA Rules not processed for user `10c0bed169344a77bfad29624c5e5a8e`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:51:11.964240 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-b10f5aa3-4128-4b08-8f69-6db3a1489ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:51:11.981488 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:51:12.036845 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 752/1518] 10.222.0.33 () {58 vars in 1015 bytes} [Thu Jul 21 22:51:11 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:51:16.037874 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1addb0c-c086-4e7c-b8e3-fda57c9d58b2 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:51:16.038391 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1addb0c-c086-4e7c-b8e3-fda57c9d58b2 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:51:16.038391 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1addb0c-c086-4e7c-b8e3-fda57c9d58b2 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:51:16.039741 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 767/1519] 10.222.0.52 () {54 vars in 961 bytes} [Thu Jul 21 22:51:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:51:16.044968 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41455d-77e2-49c9-898d-651b27bc143e None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:51:16.045384 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41455d-77e2-49c9-898d-651b27bc143e None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:51:16.045523 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41455d-77e2-49c9-898d-651b27bc143e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:51:16.063815 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-1d41455d-77e2-49c9-898d-651b27bc143e None None] MFA Rules not processed for user `10c0bed169344a77bfad29624c5e5a8e`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:51:16.065012 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-1d41455d-77e2-49c9-898d-651b27bc143e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:51:16.073535 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 753/1520] 10.222.0.52 () {58 vars in 1058 bytes} [Thu Jul 21 22:51:16 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:51:16.081625 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1addb0c-c086-4e7c-b8e3-fda57c9d58b2 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:51:16.094924 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1addb0c-c086-4e7c-b8e3-fda57c9d58b2 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:51:16.100135 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447c9086-5216-4bfc-92b2-896847783276 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10c0bed169344a77bfad29624c5e5a8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:51:16.100767 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c9086-5216-4bfc-92b2-896847783276 service placement] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:51:16.101052 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c9086-5216-4bfc-92b2-896847783276 service placement] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:51:16.101330 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c9086-5216-4bfc-92b2-896847783276 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:51:16.109733 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447c9086-5216-4bfc-92b2-896847783276 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:51:16.111328 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447c9086-5216-4bfc-92b2-896847783276 service placement] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:51:16.120807 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 768/1521] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:51:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:51:22.465875 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353da110-7221-4c6b-a9bc-23328b33c8fa None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:51:22.466500 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353da110-7221-4c6b-a9bc-23328b33c8fa None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:51:22.466500 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353da110-7221-4c6b-a9bc-23328b33c8fa None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:51:22.467931 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 754/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 22:51:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:51:22.473107 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75126dd1-ff8a-4a49-84e0-d2fe41c6a967 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:51:22.473556 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75126dd1-ff8a-4a49-84e0-d2fe41c6a967 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:51:22.473658 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75126dd1-ff8a-4a49-84e0-d2fe41c6a967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:51:22.492859 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-75126dd1-ff8a-4a49-84e0-d2fe41c6a967 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:51:22.494072 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-75126dd1-ff8a-4a49-84e0-d2fe41c6a967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:51:22.503106 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 769/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 22:51:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:51:22.660941 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353da110-7221-4c6b-a9bc-23328b33c8fa None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:51:22.675211 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353da110-7221-4c6b-a9bc-23328b33c8fa None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:51:22.680742 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b375a7b1-35b0-4d41-ae71-1ab7d4878ec9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bfdb655a2a44b33ab724daaf73a294b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:51:22.681385 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b375a7b1-35b0-4d41-ae71-1ab7d4878ec9 service nova] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:51:22.681669 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b375a7b1-35b0-4d41-ae71-1ab7d4878ec9 service nova] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:51:22.681943 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b375a7b1-35b0-4d41-ae71-1ab7d4878ec9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:51:22.690983 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b375a7b1-35b0-4d41-ae71-1ab7d4878ec9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:51:22.692595 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b375a7b1-35b0-4d41-ae71-1ab7d4878ec9 service nova] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:51:22.702604 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 755/1524] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:51:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:51:23.523308 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94479240-8403-4f01-b3a8-d3c500822cf1 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:51:23.523770 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94479240-8403-4f01-b3a8-d3c500822cf1 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:51:23.523894 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94479240-8403-4f01-b3a8-d3c500822cf1 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:51:23.525214 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 770/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 22:51:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:51:23.530003 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1dcf1b-04b0-43f3-860c-356c03a6fc13 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:51:23.530426 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1dcf1b-04b0-43f3-860c-356c03a6fc13 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:51:23.530560 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1dcf1b-04b0-43f3-860c-356c03a6fc13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:51:23.548282 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-ce1dcf1b-04b0-43f3-860c-356c03a6fc13 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:51:23.549474 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-ce1dcf1b-04b0-43f3-860c-356c03a6fc13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:51:23.557312 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 756/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 22:51:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:51:23.711494 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94479240-8403-4f01-b3a8-d3c500822cf1 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:51:23.726042 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94479240-8403-4f01-b3a8-d3c500822cf1 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:51:23.731551 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9018f3f-03e9-44dc-b54a-5762a5e1bb5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bfdb655a2a44b33ab724daaf73a294b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:51:23.732209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9018f3f-03e9-44dc-b54a-5762a5e1bb5d service nova] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:51:23.732496 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9018f3f-03e9-44dc-b54a-5762a5e1bb5d service nova] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:51:23.732775 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9018f3f-03e9-44dc-b54a-5762a5e1bb5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:51:23.740837 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9018f3f-03e9-44dc-b54a-5762a5e1bb5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:51:23.742417 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9018f3f-03e9-44dc-b54a-5762a5e1bb5d service nova] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:51:23.752384 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 771/1527] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:51:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:51:27.804098 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c094e6-50fc-4841-a66e-a75211d8f702 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:51:27.804605 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c094e6-50fc-4841-a66e-a75211d8f702 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:51:27.804605 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c094e6-50fc-4841-a66e-a75211d8f702 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:51:27.805998 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 757/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 22:51:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:51:27.811161 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db3483-37d0-4d0f-ad84-4c2f58b66ed9 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:51:27.811574 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db3483-37d0-4d0f-ad84-4c2f58b66ed9 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:51:27.811723 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db3483-37d0-4d0f-ad84-4c2f58b66ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:51:27.829772 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-b0db3483-37d0-4d0f-ad84-4c2f58b66ed9 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:51:27.830987 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-b0db3483-37d0-4d0f-ad84-4c2f58b66ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:51:27.838743 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 772/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 22:51:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:51:27.917770 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c094e6-50fc-4841-a66e-a75211d8f702 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:51:27.919896 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-49c094e6-50fc-4841-a66e-a75211d8f702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:51:28.000532 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c094e6-50fc-4841-a66e-a75211d8f702 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:51:28.004241 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150483a-b20b-4652-9306-0df524d51c3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:51:28.004884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150483a-b20b-4652-9306-0df524d51c3e service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:51:28.005164 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150483a-b20b-4652-9306-0df524d51c3e service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:51:28.005437 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150483a-b20b-4652-9306-0df524d51c3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:51:28.015127 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3150483a-b20b-4652-9306-0df524d51c3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:51:28.016549 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3150483a-b20b-4652-9306-0df524d51c3e service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:51:28.025700 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 758/1530] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:51:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:51:30.074653 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8dd62-0df9-4e16-ab9a-6301493224df None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:51:30.075250 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8dd62-0df9-4e16-ab9a-6301493224df None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:51:30.075250 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8dd62-0df9-4e16-ab9a-6301493224df None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:51:30.076596 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 773/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 22:51:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:51:30.081832 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6c8d7-865a-477e-ba67-8e5dfd3600a0 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:51:30.082275 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6c8d7-865a-477e-ba67-8e5dfd3600a0 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:51:30.082381 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6c8d7-865a-477e-ba67-8e5dfd3600a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:51:30.100201 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-09a6c8d7-865a-477e-ba67-8e5dfd3600a0 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:51:30.101563 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-09a6c8d7-865a-477e-ba67-8e5dfd3600a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:51:30.109972 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 759/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 22:51:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:51:30.146823 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df8dd62-0df9-4e16-ab9a-6301493224df None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:51:30.157778 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df8dd62-0df9-4e16-ab9a-6301493224df None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:51:30.161382 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bc568a-ff61-4cad-8434-de95ff2b8d64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:51:30.162020 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bc568a-ff61-4cad-8434-de95ff2b8d64 service neutron] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:51:30.162306 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bc568a-ff61-4cad-8434-de95ff2b8d64 service neutron] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:51:30.162582 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bc568a-ff61-4cad-8434-de95ff2b8d64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:51:30.172198 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bc568a-ff61-4cad-8434-de95ff2b8d64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:51:30.173620 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bc568a-ff61-4cad-8434-de95ff2b8d64 service neutron] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:51:30.183186 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 774/1533] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:51:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:51:34.109732 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f4923-cbc6-4b05-ac67-72e01246ff40 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:51:34.110264 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f4923-cbc6-4b05-ac67-72e01246ff40 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:51:34.110264 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f4923-cbc6-4b05-ac67-72e01246ff40 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:51:34.111756 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 760/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 22:51:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:51:34.117191 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57098e-3e1e-4927-a71b-7467be9fcdbe None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:51:34.117619 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57098e-3e1e-4927-a71b-7467be9fcdbe None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:51:34.117748 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57098e-3e1e-4927-a71b-7467be9fcdbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:51:34.124629 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=81 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:51:34.136443 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-6e57098e-3e1e-4927-a71b-7467be9fcdbe None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:51:34.138305 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-6e57098e-3e1e-4927-a71b-7467be9fcdbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:51:34.146099 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 775/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 22:51:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:51:34.450110 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3f4923-cbc6-4b05-ac67-72e01246ff40 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:51:34.451810 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-cf3f4923-cbc6-4b05-ac67-72e01246ff40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:51:34.518020 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:51:34.531808 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3f4923-cbc6-4b05-ac67-72e01246ff40 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:51:34.536914 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4061d3f2-b440-4ec3-b465-ea8e6e30687f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:51:34.537561 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061d3f2-b440-4ec3-b465-ea8e6e30687f service cinder] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:51:34.537840 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061d3f2-b440-4ec3-b465-ea8e6e30687f service cinder] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:51:34.538117 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061d3f2-b440-4ec3-b465-ea8e6e30687f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:51:34.556839 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4061d3f2-b440-4ec3-b465-ea8e6e30687f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:51:34.559035 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4061d3f2-b440-4ec3-b465-ea8e6e30687f service cinder] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:51:34.570019 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 761/1536] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:51:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:51:36.661864 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ff366-a461-4cda-ab86-34c605d64fe5 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:51:36.662419 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ff366-a461-4cda-ab86-34c605d64fe5 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:51:36.662419 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ff366-a461-4cda-ab86-34c605d64fe5 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:51:36.663810 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 776/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 22:51:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:51:36.669248 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98478d27-40ed-4b16-a2ed-cde436236b15 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:51:36.669670 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98478d27-40ed-4b16-a2ed-cde436236b15 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:51:36.669804 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98478d27-40ed-4b16-a2ed-cde436236b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:51:36.687985 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-98478d27-40ed-4b16-a2ed-cde436236b15 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:51:36.689183 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-98478d27-40ed-4b16-a2ed-cde436236b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:51:36.696937 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 762/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 22:51:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:51:36.999256 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24ff366-a461-4cda-ab86-34c605d64fe5 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:51:37.001025 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-c24ff366-a461-4cda-ab86-34c605d64fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:51:37.014698 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24ff366-a461-4cda-ab86-34c605d64fe5 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:51:37.018170 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ca3802-57c7-4c19-80ac-5e0d5a6b2a77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:51:37.018832 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca3802-57c7-4c19-80ac-5e0d5a6b2a77 service cinder] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:51:37.019118 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca3802-57c7-4c19-80ac-5e0d5a6b2a77 service cinder] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:51:37.019394 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca3802-57c7-4c19-80ac-5e0d5a6b2a77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:51:37.028239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ca3802-57c7-4c19-80ac-5e0d5a6b2a77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:51:37.029644 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ca3802-57c7-4c19-80ac-5e0d5a6b2a77 service cinder] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:51:37.038886 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 777/1539] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:51:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:51:46.682519 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=11 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:51:47.034816 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=8 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:05.720935 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb2d842-8768-4540-b4eb-b46dc012db45 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:05.721588 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb2d842-8768-4540-b4eb-b46dc012db45 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:05.721588 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb2d842-8768-4540-b4eb-b46dc012db45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:05.744869 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-acb2d842-8768-4540-b4eb-b46dc012db45 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:05.746105 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-acb2d842-8768-4540-b4eb-b46dc012db45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:05.760043 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 763/1540] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:05.769151 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ade7d07-8d7b-44be-ba75-36f3b894a899 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:05.771612 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ade7d07-8d7b-44be-ba75-36f3b894a899 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:05.771612 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ade7d07-8d7b-44be-ba75-36f3b894a899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:05.798129 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-8ade7d07-8d7b-44be-ba75-36f3b894a899 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:05.799862 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-8ade7d07-8d7b-44be-ba75-36f3b894a899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:05.809580 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba705249-0750-4cac-913d-4d648746d8db None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:05.810232 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba705249-0750-4cac-913d-4d648746d8db None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:05.810737 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba705249-0750-4cac-913d-4d648746d8db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:05.812994 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 778/1541] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:05.818438 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ade7d07-8d7b-44be-ba75-36f3b894a899 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:05.828921 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ade7d07-8d7b-44be-ba75-36f3b894a899 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:05.841654 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5739f3-58b5-41a5-9506-9f37c7057f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:05.842644 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5739f3-58b5-41a5-9506-9f37c7057f27 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:05.843115 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5739f3-58b5-41a5-9506-9f37c7057f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:05.843548 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5739f3-58b5-41a5-9506-9f37c7057f27 admin admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:05.844731 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5739f3-58b5-41a5-9506-9f37c7057f27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:53:05.845008 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-ba705249-0750-4cac-913d-4d648746d8db None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:05.845386 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5739f3-58b5-41a5-9506-9f37c7057f27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:05.846464 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-ba705249-0750-4cac-913d-4d648746d8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:05.846816 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: WARNING py.warnings [None req-7f5739f3-58b5-41a5-9506-9f37c7057f27 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 21 22:53:05.846816 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: warnings.warn(msg) Jul 21 22:53:05.846816 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: Jul 21 22:53:05.848064 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5739f3-58b5-41a5-9506-9f37c7057f27 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:05.863621 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 764/1542] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:05.871103 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 779/1543] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 22:53:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:53:05.872405 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1423554-5344-47b8-ae5c-2eab22a97dd8 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:05.872794 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1423554-5344-47b8-ae5c-2eab22a97dd8 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:05.873341 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1423554-5344-47b8-ae5c-2eab22a97dd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:05.888139 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5739f3-58b5-41a5-9506-9f37c7057f27 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:05.902296 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5739f3-58b5-41a5-9506-9f37c7057f27 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:05.912059 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-e1423554-5344-47b8-ae5c-2eab22a97dd8 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:05.913327 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-e1423554-5344-47b8-ae5c-2eab22a97dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:05.916244 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de951c87-8d9b-4f48-a43c-b3c6fd4b48b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:05.917305 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de951c87-8d9b-4f48-a43c-b3c6fd4b48b1 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:05.917995 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de951c87-8d9b-4f48-a43c-b3c6fd4b48b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:05.918494 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de951c87-8d9b-4f48-a43c-b3c6fd4b48b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:05.919486 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de951c87-8d9b-4f48-a43c-b3c6fd4b48b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:05.921635 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de951c87-8d9b-4f48-a43c-b3c6fd4b48b1 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:05.939845 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 765/1544] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:05.947217 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1423554-5344-47b8-ae5c-2eab22a97dd8 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:05.969854 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1423554-5344-47b8-ae5c-2eab22a97dd8 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:05.985207 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9608d1d7-bcb7-47c1-8e6e-b760482ad0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:05.985884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9608d1d7-bcb7-47c1-8e6e-b760482ad0dd admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:05.986167 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9608d1d7-bcb7-47c1-8e6e-b760482ad0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:05.986687 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9608d1d7-bcb7-47c1-8e6e-b760482ad0dd admin admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:05.987456 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9608d1d7-bcb7-47c1-8e6e-b760482ad0dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:53:05.987915 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9608d1d7-bcb7-47c1-8e6e-b760482ad0dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:05.997087 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9608d1d7-bcb7-47c1-8e6e-b760482ad0dd admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:06.014568 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 766/1545] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 22:53:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:53:06.021978 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9608d1d7-bcb7-47c1-8e6e-b760482ad0dd admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:06.037367 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9608d1d7-bcb7-47c1-8e6e-b760482ad0dd admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:06.043071 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f209a-3f47-4266-a279-034408497972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:06.048319 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f209a-3f47-4266-a279-034408497972 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:06.048319 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f209a-3f47-4266-a279-034408497972 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:06.048319 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f209a-3f47-4266-a279-034408497972 admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:06.048319 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587f209a-3f47-4266-a279-034408497972 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:06.048319 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: WARNING py.warnings [None req-587f209a-3f47-4266-a279-034408497972 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 22:53:06.048319 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: warnings.warn(msg) Jul 21 22:53:06.048319 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: Jul 21 22:53:06.048856 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587f209a-3f47-4266-a279-034408497972 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:06.089994 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 780/1546] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:53:05 2022] POST /identity/v3/projects => generated 361 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:53:06.095331 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de951c87-8d9b-4f48-a43c-b3c6fd4b48b1 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:06.164113 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 767/1547] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:53:06 2022] POST /identity/v3/projects => generated 367 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:53:06.167105 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68315a86-9a7b-49b1-b4ad-5f03ca6e868f None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:06.167400 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68315a86-9a7b-49b1-b4ad-5f03ca6e868f None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:06.167680 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68315a86-9a7b-49b1-b4ad-5f03ca6e868f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:06.185504 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de951c87-8d9b-4f48-a43c-b3c6fd4b48b1 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:06.211896 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-68315a86-9a7b-49b1-b4ad-5f03ca6e868f None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:06.213433 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-68315a86-9a7b-49b1-b4ad-5f03ca6e868f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:06.300376 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a499008-2c22-477e-b416-244089a4e341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:06.301601 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a499008-2c22-477e-b416-244089a4e341 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:06.302104 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a499008-2c22-477e-b416-244089a4e341 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:06.302594 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a499008-2c22-477e-b416-244089a4e341 admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:06.303676 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a499008-2c22-477e-b416-244089a4e341 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:06.304906 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: WARNING py.warnings [None req-5a499008-2c22-477e-b416-244089a4e341 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 22:53:06.304906 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: warnings.warn(msg) Jul 21 22:53:06.304906 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: Jul 21 22:53:06.307069 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a499008-2c22-477e-b416-244089a4e341 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:06.326715 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 768/1548] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:06.328142 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68315a86-9a7b-49b1-b4ad-5f03ca6e868f None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:06.343727 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68315a86-9a7b-49b1-b4ad-5f03ca6e868f None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:06.349780 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b64e599-5b8a-4d89-bbed-5f767a7058a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:06.350539 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64e599-5b8a-4d89-bbed-5f767a7058a9 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:06.350829 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64e599-5b8a-4d89-bbed-5f767a7058a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:06.351111 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64e599-5b8a-4d89-bbed-5f767a7058a9 admin admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:06.351799 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b64e599-5b8a-4d89-bbed-5f767a7058a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:06.353771 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b64e599-5b8a-4d89-bbed-5f767a7058a9 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:06.426762 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 781/1549] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:53:06 2022] POST /identity/v3/users => generated 281 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:53:06.428075 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d83ca0d-479d-4e2c-a32f-014c4acbe6cb None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:06.428368 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d83ca0d-479d-4e2c-a32f-014c4acbe6cb None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:06.428649 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d83ca0d-479d-4e2c-a32f-014c4acbe6cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:06.447128 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-9d83ca0d-479d-4e2c-a32f-014c4acbe6cb None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:06.448332 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-9d83ca0d-479d-4e2c-a32f-014c4acbe6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:06.456174 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 782/1550] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:06.457322 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d812cd0-ca84-4a9e-9ec6-91922c27842f None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:06.457612 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d812cd0-ca84-4a9e-9ec6-91922c27842f None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:06.457892 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d812cd0-ca84-4a9e-9ec6-91922c27842f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:06.471623 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 769/1551] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:53:06 2022] POST /identity/v3/users => generated 284 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:53:06.473927 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e38c8-8b3e-41b1-9462-81e5dbcb443b None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:06.473927 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e38c8-8b3e-41b1-9462-81e5dbcb443b None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:06.473927 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e38c8-8b3e-41b1-9462-81e5dbcb443b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:06.477121 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-6d812cd0-ca84-4a9e-9ec6-91922c27842f None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:06.478343 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-6d812cd0-ca84-4a9e-9ec6-91922c27842f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:06.486424 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 783/1552] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:06.487707 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346bb23-b455-492f-8436-60c11daeb12a None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:06.487978 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346bb23-b455-492f-8436-60c11daeb12a None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:06.488257 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346bb23-b455-492f-8436-60c11daeb12a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:06.501212 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-762e38c8-8b3e-41b1-9462-81e5dbcb443b None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:06.502430 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-762e38c8-8b3e-41b1-9462-81e5dbcb443b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:06.506736 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-9346bb23-b455-492f-8436-60c11daeb12a None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:06.507917 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-9346bb23-b455-492f-8436-60c11daeb12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:06.511188 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 770/1553] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:06.515923 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8774aaa4-84ee-454a-a3f4-edf0207c4d8f None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:06.516230 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8774aaa4-84ee-454a-a3f4-edf0207c4d8f None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:06.516477 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 784/1554] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:06.518995 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8774aaa4-84ee-454a-a3f4-edf0207c4d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:06.519091 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33f971-5a6a-4d51-8894-6c01bd0982f8 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:06.519091 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33f971-5a6a-4d51-8894-6c01bd0982f8 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:06.519091 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33f971-5a6a-4d51-8894-6c01bd0982f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:06.535505 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-8774aaa4-84ee-454a-a3f4-edf0207c4d8f None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:06.536731 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-8774aaa4-84ee-454a-a3f4-edf0207c4d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:06.539935 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-4c33f971-5a6a-4d51-8894-6c01bd0982f8 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:06.541067 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-4c33f971-5a6a-4d51-8894-6c01bd0982f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:06.545083 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 771/1555] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:06.545844 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8774aaa4-84ee-454a-a3f4-edf0207c4d8f None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:06.549142 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 785/1556] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:06.550306 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54549ab-4c59-4db7-8ab7-7e1f3e1d41f1 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:06.550649 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54549ab-4c59-4db7-8ab7-7e1f3e1d41f1 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:06.550899 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54549ab-4c59-4db7-8ab7-7e1f3e1d41f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:06.562134 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8774aaa4-84ee-454a-a3f4-edf0207c4d8f None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:06.566891 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc19719a-82ed-41a1-9c22-0a0b58950d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:06.567565 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc19719a-82ed-41a1-9c22-0a0b58950d2a admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:06.567847 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc19719a-82ed-41a1-9c22-0a0b58950d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:06.568121 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc19719a-82ed-41a1-9c22-0a0b58950d2a admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:06.568850 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc19719a-82ed-41a1-9c22-0a0b58950d2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:06.570436 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc19719a-82ed-41a1-9c22-0a0b58950d2a admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:06.571538 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-f54549ab-4c59-4db7-8ab7-7e1f3e1d41f1 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:06.572715 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-f54549ab-4c59-4db7-8ab7-7e1f3e1d41f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:06.580740 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 786/1557] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:06.581415 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 772/1558] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:53:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:53:06.582534 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000167aa-3603-433c-b907-bd84afc50566 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:06.582856 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000167aa-3603-433c-b907-bd84afc50566 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:06.583139 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000167aa-3603-433c-b907-bd84afc50566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:06.583526 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54549ab-4c59-4db7-8ab7-7e1f3e1d41f1 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:06.594467 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54549ab-4c59-4db7-8ab7-7e1f3e1d41f1 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:06.599197 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf310f8b-a136-4d1d-9dee-24da2b78ae6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:06.599842 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf310f8b-a136-4d1d-9dee-24da2b78ae6e admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:06.600120 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf310f8b-a136-4d1d-9dee-24da2b78ae6e admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:06.600394 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf310f8b-a136-4d1d-9dee-24da2b78ae6e admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:06.601109 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf310f8b-a136-4d1d-9dee-24da2b78ae6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:06.601825 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-000167aa-3603-433c-b907-bd84afc50566 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:06.602682 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf310f8b-a136-4d1d-9dee-24da2b78ae6e admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:06.603130 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-000167aa-3603-433c-b907-bd84afc50566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:06.611094 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 773/1559] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:06.611851 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000167aa-3603-433c-b907-bd84afc50566 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:06.615543 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 787/1560] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:53:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:53:06.615543 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9769fa-c787-46f5-90d3-45de6337057f None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:06.615543 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9769fa-c787-46f5-90d3-45de6337057f None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:06.615543 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9769fa-c787-46f5-90d3-45de6337057f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:06.620607 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:06.624905 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000167aa-3603-433c-b907-bd84afc50566 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:06.629488 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e833e5-9e1e-4327-b55e-d2d57d3b8b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:06.630136 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e833e5-9e1e-4327-b55e-d2d57d3b8b19 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:06.630416 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e833e5-9e1e-4327-b55e-d2d57d3b8b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:06.634238 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e833e5-9e1e-4327-b55e-d2d57d3b8b19 admin admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:06.634238 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e833e5-9e1e-4327-b55e-d2d57d3b8b19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:53:06.634238 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e833e5-9e1e-4327-b55e-d2d57d3b8b19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:06.634525 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-af9769fa-c787-46f5-90d3-45de6337057f None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:06.634525 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-af9769fa-c787-46f5-90d3-45de6337057f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:06.634677 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e833e5-9e1e-4327-b55e-d2d57d3b8b19 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:06.641687 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 788/1561] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:06.643547 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eefc772-9c79-400b-a771-0f8ffa5889d6 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:06.643957 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eefc772-9c79-400b-a771-0f8ffa5889d6 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:06.644124 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eefc772-9c79-400b-a771-0f8ffa5889d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:06.646473 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 774/1562] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 22:53:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:53:06.647674 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c71f63-d85a-4d0e-a647-d370d21d074a None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:06.647970 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c71f63-d85a-4d0e-a647-d370d21d074a None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:06.648250 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c71f63-d85a-4d0e-a647-d370d21d074a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:06.663516 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-6eefc772-9c79-400b-a771-0f8ffa5889d6 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:06.668328 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-6eefc772-9c79-400b-a771-0f8ffa5889d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:06.670414 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-87c71f63-d85a-4d0e-a647-d370d21d074a None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:06.671619 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-87c71f63-d85a-4d0e-a647-d370d21d074a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:06.672639 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 789/1563] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:06.673469 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eefc772-9c79-400b-a771-0f8ffa5889d6 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:06.684935 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eefc772-9c79-400b-a771-0f8ffa5889d6 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:06.685237 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 775/1564] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:06.686318 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c71f63-d85a-4d0e-a647-d370d21d074a None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:06.690066 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601aafd5-46bc-4d27-98de-b22b7ed43f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:06.690717 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601aafd5-46bc-4d27-98de-b22b7ed43f75 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:06.691038 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601aafd5-46bc-4d27-98de-b22b7ed43f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:06.691305 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601aafd5-46bc-4d27-98de-b22b7ed43f75 admin admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:06.692022 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601aafd5-46bc-4d27-98de-b22b7ed43f75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:53:06.692329 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601aafd5-46bc-4d27-98de-b22b7ed43f75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:06.693911 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601aafd5-46bc-4d27-98de-b22b7ed43f75 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:06.696681 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c71f63-d85a-4d0e-a647-d370d21d074a None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:06.701615 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a2732-3b18-488e-9ee2-8d2e7ef794b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:06.702271 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a2732-3b18-488e-9ee2-8d2e7ef794b5 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:06.702554 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a2732-3b18-488e-9ee2-8d2e7ef794b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:06.702864 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a2732-3b18-488e-9ee2-8d2e7ef794b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:06.703965 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047a2732-3b18-488e-9ee2-8d2e7ef794b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:06.705154 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047a2732-3b18-488e-9ee2-8d2e7ef794b5 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:06.707013 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 790/1565] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 22:53:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:53:06.708153 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601aafd5-46bc-4d27-98de-b22b7ed43f75 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:06.716519 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 776/1566] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:53:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:53:06.716708 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a2732-3b18-488e-9ee2-8d2e7ef794b5 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:06.720589 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601aafd5-46bc-4d27-98de-b22b7ed43f75 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:06.725411 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cc7dcb-465c-4fb0-90cb-4e6bbaef9f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:06.726054 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc7dcb-465c-4fb0-90cb-4e6bbaef9f50 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:06.726296 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a2732-3b18-488e-9ee2-8d2e7ef794b5 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:06.726747 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc7dcb-465c-4fb0-90cb-4e6bbaef9f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:06.726747 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc7dcb-465c-4fb0-90cb-4e6bbaef9f50 admin admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:06.727480 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cc7dcb-465c-4fb0-90cb-4e6bbaef9f50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:53:06.727787 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cc7dcb-465c-4fb0-90cb-4e6bbaef9f50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:06.729384 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cc7dcb-465c-4fb0-90cb-4e6bbaef9f50 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:06.731023 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f891341-987c-4e83-bbe8-754a325789e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:06.731661 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f891341-987c-4e83-bbe8-754a325789e5 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:06.731945 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f891341-987c-4e83-bbe8-754a325789e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:06.732222 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f891341-987c-4e83-bbe8-754a325789e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:06.732941 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f891341-987c-4e83-bbe8-754a325789e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:06.734513 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f891341-987c-4e83-bbe8-754a325789e5 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:06.741817 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 791/1567] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 22:53:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:53:06.742695 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cc7dcb-465c-4fb0-90cb-4e6bbaef9f50 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:06.744839 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 777/1568] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:53:06 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:53:06.745492 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f891341-987c-4e83-bbe8-754a325789e5 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:06.755324 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f891341-987c-4e83-bbe8-754a325789e5 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:06.756622 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cc7dcb-465c-4fb0-90cb-4e6bbaef9f50 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:06.760431 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f37aa50-60ed-45d1-9038-50de8350088c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:06.761062 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f37aa50-60ed-45d1-9038-50de8350088c admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:06.761345 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f37aa50-60ed-45d1-9038-50de8350088c admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:06.761625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f37aa50-60ed-45d1-9038-50de8350088c admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:06.761808 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f6f29c-4ed8-4fb0-80b0-64fb679b741a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:06.762300 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f37aa50-60ed-45d1-9038-50de8350088c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:06.762446 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f6f29c-4ed8-4fb0-80b0-64fb679b741a admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:06.762726 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f6f29c-4ed8-4fb0-80b0-64fb679b741a admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:06.763063 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f6f29c-4ed8-4fb0-80b0-64fb679b741a admin admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:06.763777 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f6f29c-4ed8-4fb0-80b0-64fb679b741a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:53:06.764086 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f6f29c-4ed8-4fb0-80b0-64fb679b741a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:06.764270 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f37aa50-60ed-45d1-9038-50de8350088c admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:06.765668 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f6f29c-4ed8-4fb0-80b0-64fb679b741a admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:06.778089 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 792/1569] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 22:53:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:53:06.787770 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:06.787898 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f6f29c-4ed8-4fb0-80b0-64fb679b741a admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:06.788397 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f6f29c-4ed8-4fb0-80b0-64fb679b741a admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:06.793151 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f0ca5-c546-4648-b072-f6bc4f8e4989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:06.793794 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f0ca5-c546-4648-b072-f6bc4f8e4989 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:06.794080 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f0ca5-c546-4648-b072-f6bc4f8e4989 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:06.794362 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f0ca5-c546-4648-b072-f6bc4f8e4989 admin admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:06.795143 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08f0ca5-c546-4648-b072-f6bc4f8e4989 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:53:06.795458 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08f0ca5-c546-4648-b072-f6bc4f8e4989 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:06.797131 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08f0ca5-c546-4648-b072-f6bc4f8e4989 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:06.812066 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 793/1570] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 22:53:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:53:06.812066 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f0ca5-c546-4648-b072-f6bc4f8e4989 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:06.820090 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f0ca5-c546-4648-b072-f6bc4f8e4989 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:06.825463 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24352759-8efe-49df-994d-691fa67ff93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:06.826084 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24352759-8efe-49df-994d-691fa67ff93f admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:06.826412 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24352759-8efe-49df-994d-691fa67ff93f admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:06.826695 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24352759-8efe-49df-994d-691fa67ff93f admin admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:06.827423 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24352759-8efe-49df-994d-691fa67ff93f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:53:06.827896 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24352759-8efe-49df-994d-691fa67ff93f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:06.829596 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24352759-8efe-49df-994d-691fa67ff93f admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:06.841885 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 794/1571] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 22:53:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:53:06.842632 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24352759-8efe-49df-994d-691fa67ff93f admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:06.852232 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24352759-8efe-49df-994d-691fa67ff93f admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:06.856763 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b80519-9afd-4e4f-9145-f4cddc0dd2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:06.857377 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b80519-9afd-4e4f-9145-f4cddc0dd2d4 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:06.857660 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b80519-9afd-4e4f-9145-f4cddc0dd2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:06.857942 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b80519-9afd-4e4f-9145-f4cddc0dd2d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:06.858607 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b80519-9afd-4e4f-9145-f4cddc0dd2d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:06.860519 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b80519-9afd-4e4f-9145-f4cddc0dd2d4 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:06.866580 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 778/1572] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:53:06 2022] POST /identity/v3/projects => generated 373 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:53:06.867420 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f37aa50-60ed-45d1-9038-50de8350088c admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:06.927419 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f37aa50-60ed-45d1-9038-50de8350088c admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:06.932554 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a656c40-1306-42d4-b14a-cdd2175968ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:06.933436 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a656c40-1306-42d4-b14a-cdd2175968ff admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:06.933714 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a656c40-1306-42d4-b14a-cdd2175968ff admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:06.933987 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a656c40-1306-42d4-b14a-cdd2175968ff admin admin] PATH_INFO: `/v3/projects/faadfc4e89d649c1b54da92755178fee/users/d9bcda22585e4e6ea8bf19b162406566/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:06.950515 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a656c40-1306-42d4-b14a-cdd2175968ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=faadfc4e89d649c1b54da92755178fee, user_id=d9bcda22585e4e6ea8bf19b162406566, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:06.951589 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: WARNING py.warnings [None req-4a656c40-1306-42d4-b14a-cdd2175968ff admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 22:53:06.951589 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: warnings.warn(msg) Jul 21 22:53:06.951589 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: Jul 21 22:53:06.952456 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 795/1573] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:53:06 2022] POST /identity/v3/projects => generated 391 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:53:06.953276 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b80519-9afd-4e4f-9145-f4cddc0dd2d4 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:06.953530 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a656c40-1306-42d4-b14a-cdd2175968ff admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:07.016245 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b80519-9afd-4e4f-9145-f4cddc0dd2d4 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:07.020982 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfeedee-729f-4826-bd7e-fc38cc23f680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:07.021640 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfeedee-729f-4826-bd7e-fc38cc23f680 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:07.021932 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfeedee-729f-4826-bd7e-fc38cc23f680 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:07.022220 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfeedee-729f-4826-bd7e-fc38cc23f680 admin admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:07.022944 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfeedee-729f-4826-bd7e-fc38cc23f680 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:07.024839 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfeedee-729f-4826-bd7e-fc38cc23f680 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:07.026183 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 779/1574] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:53:06 2022] PUT /identity/v3/projects/faadfc4e89d649c1b54da92755178fee/users/d9bcda22585e4e6ea8bf19b162406566/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:07.027081 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a656c40-1306-42d4-b14a-cdd2175968ff admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:07.093934 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a656c40-1306-42d4-b14a-cdd2175968ff admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:07.098670 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1744c484-dafb-434f-8bcc-d620e213a77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:07.099629 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744c484-dafb-434f-8bcc-d620e213a77b admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:07.099963 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744c484-dafb-434f-8bcc-d620e213a77b admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:07.100242 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744c484-dafb-434f-8bcc-d620e213a77b admin admin] PATH_INFO: `/v3/projects/cdb170fd25234969b1ab3e7b828dca86/users/57db3b9e6fe642c9ad51f10dfc62c59f/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:07.107889 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 796/1575] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:53:06 2022] POST /identity/v3/projects => generated 379 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:53:07.108711 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfeedee-729f-4826-bd7e-fc38cc23f680 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:07.120373 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1744c484-dafb-434f-8bcc-d620e213a77b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdb170fd25234969b1ab3e7b828dca86, user_id=57db3b9e6fe642c9ad51f10dfc62c59f, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:07.123203 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1744c484-dafb-434f-8bcc-d620e213a77b admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:07.174876 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:07.186802 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 780/1576] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:53:07 2022] PUT /identity/v3/projects/cdb170fd25234969b1ab3e7b828dca86/users/57db3b9e6fe642c9ad51f10dfc62c59f/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:07.189551 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1744c484-dafb-434f-8bcc-d620e213a77b admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:07.198528 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfeedee-729f-4826-bd7e-fc38cc23f680 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:07.253752 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1744c484-dafb-434f-8bcc-d620e213a77b admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:07.257369 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09047eec-18f0-4f26-bb96-765f9c9d7452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:07.258143 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09047eec-18f0-4f26-bb96-765f9c9d7452 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:07.258925 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a0900-801e-40a7-a59b-2943ce3e692c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:07.259091 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09047eec-18f0-4f26-bb96-765f9c9d7452 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:07.259386 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09047eec-18f0-4f26-bb96-765f9c9d7452 admin admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:07.259608 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a0900-801e-40a7-a59b-2943ce3e692c admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:07.259889 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a0900-801e-40a7-a59b-2943ce3e692c admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:07.260062 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09047eec-18f0-4f26-bb96-765f9c9d7452 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:07.260212 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a0900-801e-40a7-a59b-2943ce3e692c admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:07.260830 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70a0900-801e-40a7-a59b-2943ce3e692c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:07.261967 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09047eec-18f0-4f26-bb96-765f9c9d7452 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:07.262908 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70a0900-801e-40a7-a59b-2943ce3e692c admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:07.266076 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:07.379561 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 781/1577] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:53:07 2022] POST /identity/v3/projects => generated 387 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:53:07.379561 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a0900-801e-40a7-a59b-2943ce3e692c admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:07.407782 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 797/1578] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:53:07 2022] POST /identity/v3/projects => generated 375 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:53:07.407782 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09047eec-18f0-4f26-bb96-765f9c9d7452 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:07.438569 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a0900-801e-40a7-a59b-2943ce3e692c admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:07.468478 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09047eec-18f0-4f26-bb96-765f9c9d7452 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:07.473409 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8bd7f3-09bf-4674-b25a-1d136de07452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:07.474076 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8bd7f3-09bf-4674-b25a-1d136de07452 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:07.474370 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8bd7f3-09bf-4674-b25a-1d136de07452 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:07.474644 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8bd7f3-09bf-4674-b25a-1d136de07452 admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:07.475377 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8bd7f3-09bf-4674-b25a-1d136de07452 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:07.477842 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8bd7f3-09bf-4674-b25a-1d136de07452 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:07.495338 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa817112-45ba-48ff-89c7-e9c7f48e090f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:07.497012 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa817112-45ba-48ff-89c7-e9c7f48e090f admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:07.497012 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa817112-45ba-48ff-89c7-e9c7f48e090f admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:07.497012 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa817112-45ba-48ff-89c7-e9c7f48e090f admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:07.497295 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa817112-45ba-48ff-89c7-e9c7f48e090f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:07.499223 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa817112-45ba-48ff-89c7-e9c7f48e090f admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:07.544999 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 798/1579] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:53:07 2022] POST /identity/v3/users => generated 287 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:53:07.545877 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8bd7f3-09bf-4674-b25a-1d136de07452 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:07.555944 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8bd7f3-09bf-4674-b25a-1d136de07452 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:07.560518 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43901d71-456a-4c28-a53c-378c6665c507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:07.561152 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43901d71-456a-4c28-a53c-378c6665c507 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:07.561436 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43901d71-456a-4c28-a53c-378c6665c507 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:07.561714 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43901d71-456a-4c28-a53c-378c6665c507 admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:07.562389 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43901d71-456a-4c28-a53c-378c6665c507 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:07.564320 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43901d71-456a-4c28-a53c-378c6665c507 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:07.603853 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 782/1580] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:53:07 2022] POST /identity/v3/projects => generated 385 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:53:07.605038 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd051a5-ff57-40d0-b981-c1232fb9ef28 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:07.605327 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd051a5-ff57-40d0-b981-c1232fb9ef28 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:07.605637 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd051a5-ff57-40d0-b981-c1232fb9ef28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:07.631655 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-cbd051a5-ff57-40d0-b981-c1232fb9ef28 None None] MFA Rules not processed for user `d9bcda22585e4e6ea8bf19b162406566`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:07.632794 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-cbd051a5-ff57-40d0-b981-c1232fb9ef28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:07.650097 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 799/1581] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:53:07 2022] POST /identity/v3/users => generated 296 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:53:07.651915 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43901d71-456a-4c28-a53c-378c6665c507 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:07.660861 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43901d71-456a-4c28-a53c-378c6665c507 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:07.699657 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 783/1582] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:07 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:07.700984 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83d3c01-3448-4486-aedd-5dde7b5378ec None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:07.701276 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83d3c01-3448-4486-aedd-5dde7b5378ec None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:07.701556 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83d3c01-3448-4486-aedd-5dde7b5378ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:07.712524 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:07.714198 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:07.722465 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee785a6-e293-47f7-b448-0d80161aaaef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:07.724992 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee785a6-e293-47f7-b448-0d80161aaaef admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:07.724992 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee785a6-e293-47f7-b448-0d80161aaaef admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:07.724992 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee785a6-e293-47f7-b448-0d80161aaaef admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:07.724992 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee785a6-e293-47f7-b448-0d80161aaaef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:07.726306 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee785a6-e293-47f7-b448-0d80161aaaef admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:07.729218 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-d83d3c01-3448-4486-aedd-5dde7b5378ec None None] MFA Rules not processed for user `57db3b9e6fe642c9ad51f10dfc62c59f`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:07.730469 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-d83d3c01-3448-4486-aedd-5dde7b5378ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:07.794456 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 784/1583] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:07 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:07.795446 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83d3c01-3448-4486-aedd-5dde7b5378ec None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:07.800821 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 800/1584] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:53:07 2022] POST /identity/v3/users => generated 290 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:53:07.801633 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee785a6-e293-47f7-b448-0d80161aaaef admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:07.808707 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83d3c01-3448-4486-aedd-5dde7b5378ec None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:07.813306 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee785a6-e293-47f7-b448-0d80161aaaef admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:07.813718 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e9d60d-d96b-4086-a1fb-fbba6a4f03d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:07.814413 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e9d60d-d96b-4086-a1fb-fbba6a4f03d3 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:07.814697 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e9d60d-d96b-4086-a1fb-fbba6a4f03d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:07.815008 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e9d60d-d96b-4086-a1fb-fbba6a4f03d3 admin admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:07.815703 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e9d60d-d96b-4086-a1fb-fbba6a4f03d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:07.817613 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e9d60d-d96b-4086-a1fb-fbba6a4f03d3 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:07.817876 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe5ca22-158d-433e-acd8-853e2f789f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:07.818539 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe5ca22-158d-433e-acd8-853e2f789f80 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:07.818839 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe5ca22-158d-433e-acd8-853e2f789f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:07.819118 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe5ca22-158d-433e-acd8-853e2f789f80 admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:07.819791 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe5ca22-158d-433e-acd8-853e2f789f80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:07.822325 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe5ca22-158d-433e-acd8-853e2f789f80 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:07.894667 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 785/1585] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:53:07 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:53:07.898557 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e9d60d-d96b-4086-a1fb-fbba6a4f03d3 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:07.907961 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e9d60d-d96b-4086-a1fb-fbba6a4f03d3 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:07.913427 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0930d4d8-23df-4898-b847-fa88697577e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:07.914063 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0930d4d8-23df-4898-b847-fa88697577e8 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:07.914372 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0930d4d8-23df-4898-b847-fa88697577e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:07.914657 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0930d4d8-23df-4898-b847-fa88697577e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:07.915440 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0930d4d8-23df-4898-b847-fa88697577e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:07.921342 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0930d4d8-23df-4898-b847-fa88697577e8 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:07.921465 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 801/1586] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:53:07 2022] POST /identity/v3/users => generated 288 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:53:07.921465 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe5ca22-158d-433e-acd8-853e2f789f80 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:07.928303 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 786/1587] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:53:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:53:07.929160 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0930d4d8-23df-4898-b847-fa88697577e8 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:07.929946 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe5ca22-158d-433e-acd8-853e2f789f80 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:07.935150 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190fc1fe-f61a-4516-bcb4-3eaf74f8b20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:07.935812 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190fc1fe-f61a-4516-bcb4-3eaf74f8b20c admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:07.936099 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190fc1fe-f61a-4516-bcb4-3eaf74f8b20c admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:07.936378 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190fc1fe-f61a-4516-bcb4-3eaf74f8b20c admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:07.937059 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190fc1fe-f61a-4516-bcb4-3eaf74f8b20c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:07.939030 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190fc1fe-f61a-4516-bcb4-3eaf74f8b20c admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:07.940621 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0930d4d8-23df-4898-b847-fa88697577e8 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:07.945301 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7923e937-1b9b-4a40-84c0-0ad81c508d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:07.946561 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923e937-1b9b-4a40-84c0-0ad81c508d16 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:07.946941 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923e937-1b9b-4a40-84c0-0ad81c508d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:07.947341 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923e937-1b9b-4a40-84c0-0ad81c508d16 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:07.949375 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7923e937-1b9b-4a40-84c0-0ad81c508d16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:07.950658 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7923e937-1b9b-4a40-84c0-0ad81c508d16 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:07.961261 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 787/1588] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:53:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:53:07.963014 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7923e937-1b9b-4a40-84c0-0ad81c508d16 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:07.972481 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7923e937-1b9b-4a40-84c0-0ad81c508d16 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:07.977167 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce9047a-e1bc-4634-b64f-e9ba16118cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:07.977796 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce9047a-e1bc-4634-b64f-e9ba16118cc5 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:07.978081 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce9047a-e1bc-4634-b64f-e9ba16118cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:07.978355 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce9047a-e1bc-4634-b64f-e9ba16118cc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:07.979109 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce9047a-e1bc-4634-b64f-e9ba16118cc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:07.981000 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce9047a-e1bc-4634-b64f-e9ba16118cc5 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:08.021161 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 802/1589] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:53:07 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:53:08.021973 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190fc1fe-f61a-4516-bcb4-3eaf74f8b20c admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:08.031446 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190fc1fe-f61a-4516-bcb4-3eaf74f8b20c admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:08.035992 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476904c1-930e-4839-aa37-b9aa514065a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:08.036655 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476904c1-930e-4839-aa37-b9aa514065a0 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:08.036946 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476904c1-930e-4839-aa37-b9aa514065a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:08.037225 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476904c1-930e-4839-aa37-b9aa514065a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:08.037897 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476904c1-930e-4839-aa37-b9aa514065a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:08.040353 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476904c1-930e-4839-aa37-b9aa514065a0 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:08.070504 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 788/1590] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:53:07 2022] POST /identity/v3/projects => generated 363 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:53:08.070504 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce9047a-e1bc-4634-b64f-e9ba16118cc5 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:08.125038 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 803/1591] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:53:08 2022] POST /identity/v3/projects => generated 367 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:53:08.125946 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476904c1-930e-4839-aa37-b9aa514065a0 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:08.132170 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce9047a-e1bc-4634-b64f-e9ba16118cc5 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:08.172344 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:08.188102 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476904c1-930e-4839-aa37-b9aa514065a0 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:08.190964 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78ad142-9aa2-4a43-996a-7ddf59e146b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:08.190964 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78ad142-9aa2-4a43-996a-7ddf59e146b9 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:08.190964 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78ad142-9aa2-4a43-996a-7ddf59e146b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:08.190964 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78ad142-9aa2-4a43-996a-7ddf59e146b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:08.191315 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78ad142-9aa2-4a43-996a-7ddf59e146b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:08.192329 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78ad142-9aa2-4a43-996a-7ddf59e146b9 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:08.193046 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56011e1c-bea2-41e0-bba6-7f1cbc1aca30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:08.193714 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56011e1c-bea2-41e0-bba6-7f1cbc1aca30 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:08.194000 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56011e1c-bea2-41e0-bba6-7f1cbc1aca30 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:08.194281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56011e1c-bea2-41e0-bba6-7f1cbc1aca30 admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:08.195045 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56011e1c-bea2-41e0-bba6-7f1cbc1aca30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:08.196712 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56011e1c-bea2-41e0-bba6-7f1cbc1aca30 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:08.202323 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 789/1592] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:53:08 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:53:08.203304 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78ad142-9aa2-4a43-996a-7ddf59e146b9 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:08.209981 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 804/1593] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:53:08 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:53:08.210962 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56011e1c-bea2-41e0-bba6-7f1cbc1aca30 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:08.214136 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78ad142-9aa2-4a43-996a-7ddf59e146b9 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:08.219291 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b27e067-387a-46ce-8997-9b82042a4e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:08.220069 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27e067-387a-46ce-8997-9b82042a4e2d admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:08.220357 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27e067-387a-46ce-8997-9b82042a4e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:08.220636 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27e067-387a-46ce-8997-9b82042a4e2d admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:08.220753 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56011e1c-bea2-41e0-bba6-7f1cbc1aca30 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:08.221369 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b27e067-387a-46ce-8997-9b82042a4e2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:08.223633 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b27e067-387a-46ce-8997-9b82042a4e2d admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:08.225530 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50772cf1-33d4-488b-ac84-a10c91cf6ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:08.226170 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50772cf1-33d4-488b-ac84-a10c91cf6ccb admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:08.226451 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50772cf1-33d4-488b-ac84-a10c91cf6ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:08.226721 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50772cf1-33d4-488b-ac84-a10c91cf6ccb admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:08.227995 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50772cf1-33d4-488b-ac84-a10c91cf6ccb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:08.229561 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50772cf1-33d4-488b-ac84-a10c91cf6ccb admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:08.234275 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 790/1594] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:53:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:53:08.235103 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b27e067-387a-46ce-8997-9b82042a4e2d admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:08.240613 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 805/1595] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:53:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:53:08.241828 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50772cf1-33d4-488b-ac84-a10c91cf6ccb admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:08.245498 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b27e067-387a-46ce-8997-9b82042a4e2d admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:08.250641 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55459bf-ce85-4323-ada3-fdf247118fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:08.251327 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55459bf-ce85-4323-ada3-fdf247118fa5 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:08.251612 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55459bf-ce85-4323-ada3-fdf247118fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:08.251888 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55459bf-ce85-4323-ada3-fdf247118fa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:08.252515 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50772cf1-33d4-488b-ac84-a10c91cf6ccb admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:08.252636 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55459bf-ce85-4323-ada3-fdf247118fa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:08.254183 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55459bf-ce85-4323-ada3-fdf247118fa5 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:08.257358 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a710d0-bfa7-4646-a680-8ee0b66e99a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:08.258093 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a710d0-bfa7-4646-a680-8ee0b66e99a8 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:08.258457 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a710d0-bfa7-4646-a680-8ee0b66e99a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:08.258839 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a710d0-bfa7-4646-a680-8ee0b66e99a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:08.259633 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a710d0-bfa7-4646-a680-8ee0b66e99a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:08.261196 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a710d0-bfa7-4646-a680-8ee0b66e99a8 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:08.264697 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 791/1596] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:53:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:53:08.265457 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55459bf-ce85-4323-ada3-fdf247118fa5 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:08.276430 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 806/1597] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:53:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:53:08.276430 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a710d0-bfa7-4646-a680-8ee0b66e99a8 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:08.276681 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55459bf-ce85-4323-ada3-fdf247118fa5 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:08.279764 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851a80c-15ac-4f35-a103-2b329844d684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:08.280884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851a80c-15ac-4f35-a103-2b329844d684 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:08.280884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851a80c-15ac-4f35-a103-2b329844d684 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:08.281091 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851a80c-15ac-4f35-a103-2b329844d684 admin admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:08.281626 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8851a80c-15ac-4f35-a103-2b329844d684 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:08.283557 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8851a80c-15ac-4f35-a103-2b329844d684 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:08.284080 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a710d0-bfa7-4646-a680-8ee0b66e99a8 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:08.288607 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe9dce-ba9f-4323-8b4a-7291d743154f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:08.289252 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe9dce-ba9f-4323-8b4a-7291d743154f admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:08.290154 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe9dce-ba9f-4323-8b4a-7291d743154f admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:08.290442 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe9dce-ba9f-4323-8b4a-7291d743154f admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:08.291165 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfe9dce-ba9f-4323-8b4a-7291d743154f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:08.293056 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfe9dce-ba9f-4323-8b4a-7291d743154f admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:08.369889 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 792/1598] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:53:08 2022] POST /identity/v3/users => generated 288 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:53:08.370882 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851a80c-15ac-4f35-a103-2b329844d684 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:08.380329 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851a80c-15ac-4f35-a103-2b329844d684 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:08.385064 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab2399-eae1-4f66-a719-fa2828d99523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:08.385883 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab2399-eae1-4f66-a719-fa2828d99523 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:08.386015 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab2399-eae1-4f66-a719-fa2828d99523 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:08.386234 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab2399-eae1-4f66-a719-fa2828d99523 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:08.389412 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 807/1599] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:53:08 2022] POST /identity/v3/users => generated 290 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:53:08.389559 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ab2399-eae1-4f66-a719-fa2828d99523 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:08.389665 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe9dce-ba9f-4323-8b4a-7291d743154f admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:08.389769 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ab2399-eae1-4f66-a719-fa2828d99523 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:08.397242 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe9dce-ba9f-4323-8b4a-7291d743154f admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:08.399099 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 793/1600] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:53:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:53:08.399880 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab2399-eae1-4f66-a719-fa2828d99523 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:08.405121 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8516f20b-d871-4b85-bf91-c92f30fae922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:08.405781 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8516f20b-d871-4b85-bf91-c92f30fae922 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:08.406067 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8516f20b-d871-4b85-bf91-c92f30fae922 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:08.406351 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8516f20b-d871-4b85-bf91-c92f30fae922 admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:08.407110 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8516f20b-d871-4b85-bf91-c92f30fae922 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:08.408689 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8516f20b-d871-4b85-bf91-c92f30fae922 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:08.411294 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab2399-eae1-4f66-a719-fa2828d99523 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:08.416590 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87cfe4e-d584-4903-87a5-02e435278759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:08.417228 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87cfe4e-d584-4903-87a5-02e435278759 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:08.417509 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87cfe4e-d584-4903-87a5-02e435278759 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:08.417780 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87cfe4e-d584-4903-87a5-02e435278759 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:08.418488 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87cfe4e-d584-4903-87a5-02e435278759 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:08.422669 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 808/1601] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:53:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:53:08.422833 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87cfe4e-d584-4903-87a5-02e435278759 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:08.422949 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8516f20b-d871-4b85-bf91-c92f30fae922 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:08.429557 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8516f20b-d871-4b85-bf91-c92f30fae922 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:08.431467 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 794/1602] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:53:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:53:08.432235 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87cfe4e-d584-4903-87a5-02e435278759 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:08.435206 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3120324c-fe0a-4ee0-9621-8fc4be51795f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:08.436072 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3120324c-fe0a-4ee0-9621-8fc4be51795f admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:08.436356 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3120324c-fe0a-4ee0-9621-8fc4be51795f admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:08.436638 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3120324c-fe0a-4ee0-9621-8fc4be51795f admin admin] PATH_INFO: `/v3/projects/8c0eb6e4060c4003b27ecf3401714f1d/users/d1aa30210cda4b6a9c457ba56f3a9e04/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:08.441452 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87cfe4e-d584-4903-87a5-02e435278759 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:08.446033 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fea55f7-95db-4ff7-848d-f248ec392092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:08.446906 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea55f7-95db-4ff7-848d-f248ec392092 admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:08.447187 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea55f7-95db-4ff7-848d-f248ec392092 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:08.447457 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea55f7-95db-4ff7-848d-f248ec392092 admin admin] PATH_INFO: `/v3/projects/755a70c4912f4795ae67cf9f036e02aa/users/4947466838974a71815561640fa55c14/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:08.451437 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3120324c-fe0a-4ee0-9621-8fc4be51795f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c0eb6e4060c4003b27ecf3401714f1d, user_id=d1aa30210cda4b6a9c457ba56f3a9e04, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:08.454071 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3120324c-fe0a-4ee0-9621-8fc4be51795f admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:08.462392 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fea55f7-95db-4ff7-848d-f248ec392092 admin admin] RBAC: Authorizing `identity:create_grant(project_id=755a70c4912f4795ae67cf9f036e02aa, user_id=4947466838974a71815561640fa55c14, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:08.465164 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fea55f7-95db-4ff7-848d-f248ec392092 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:08.524356 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 809/1603] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:53:08 2022] PUT /identity/v3/projects/8c0eb6e4060c4003b27ecf3401714f1d/users/d1aa30210cda4b6a9c457ba56f3a9e04/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:08.525180 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3120324c-fe0a-4ee0-9621-8fc4be51795f admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:08.530714 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=86,keystone:INSERT=14 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:08.539122 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 795/1604] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:53:08 2022] PUT /identity/v3/projects/755a70c4912f4795ae67cf9f036e02aa/users/4947466838974a71815561640fa55c14/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:08.540632 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fea55f7-95db-4ff7-848d-f248ec392092 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:08.588094 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3120324c-fe0a-4ee0-9621-8fc4be51795f admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:08.603194 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fea55f7-95db-4ff7-848d-f248ec392092 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:08.608279 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a28baa7-7f1a-4389-8184-f130000832c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:08.608987 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28baa7-7f1a-4389-8184-f130000832c7 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:08.609274 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28baa7-7f1a-4389-8184-f130000832c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:08.609553 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28baa7-7f1a-4389-8184-f130000832c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:08.610286 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a28baa7-7f1a-4389-8184-f130000832c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:08.611926 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a28baa7-7f1a-4389-8184-f130000832c7 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:08.621921 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 796/1605] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:53:08 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:53:08.625480 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a28baa7-7f1a-4389-8184-f130000832c7 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:08.632512 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a28baa7-7f1a-4389-8184-f130000832c7 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:08.637372 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e204f-2250-4a4f-90b0-20d8afc943d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:08.638259 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e204f-2250-4a4f-90b0-20d8afc943d8 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:08.638717 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e204f-2250-4a4f-90b0-20d8afc943d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:08.639096 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e204f-2250-4a4f-90b0-20d8afc943d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:08.639826 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437e204f-2250-4a4f-90b0-20d8afc943d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:08.641398 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437e204f-2250-4a4f-90b0-20d8afc943d8 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:08.645545 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46df04a6-f4a5-41a6-a149-5bd8e784d39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:08.646250 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46df04a6-f4a5-41a6-a149-5bd8e784d39e admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:08.646541 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46df04a6-f4a5-41a6-a149-5bd8e784d39e admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:08.646848 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46df04a6-f4a5-41a6-a149-5bd8e784d39e admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:08.647579 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46df04a6-f4a5-41a6-a149-5bd8e784d39e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:08.649858 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46df04a6-f4a5-41a6-a149-5bd8e784d39e admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:08.652007 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 797/1606] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:53:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:53:08.652836 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e204f-2250-4a4f-90b0-20d8afc943d8 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:08.660137 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 810/1607] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:53:08 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:53:08.660912 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46df04a6-f4a5-41a6-a149-5bd8e784d39e admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:08.666179 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e204f-2250-4a4f-90b0-20d8afc943d8 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:08.670634 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46df04a6-f4a5-41a6-a149-5bd8e784d39e admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:08.670757 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514cbd7f-1549-4323-84f4-5df1ab0c1521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:08.671616 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514cbd7f-1549-4323-84f4-5df1ab0c1521 admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:08.671899 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514cbd7f-1549-4323-84f4-5df1ab0c1521 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:08.672173 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514cbd7f-1549-4323-84f4-5df1ab0c1521 admin admin] PATH_INFO: `/v3/projects/ba0db0569e094eccb3e8fc578a16fc1e/users/ca0902f6348840729a6ce7a97167da9c/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:08.675206 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9b7d78-8a1b-4eb9-b356-ec348603e22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:08.676000 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b7d78-8a1b-4eb9-b356-ec348603e22d admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:08.676279 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b7d78-8a1b-4eb9-b356-ec348603e22d admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:08.676551 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b7d78-8a1b-4eb9-b356-ec348603e22d admin admin] PATH_INFO: `/v3/projects/2f7e8103418741398872d59d16e0e919/users/d10c68b7fd4446feb759832913fea3b3/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:08.687479 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514cbd7f-1549-4323-84f4-5df1ab0c1521 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba0db0569e094eccb3e8fc578a16fc1e, user_id=ca0902f6348840729a6ce7a97167da9c, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:08.690169 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514cbd7f-1549-4323-84f4-5df1ab0c1521 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:08.691507 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9b7d78-8a1b-4eb9-b356-ec348603e22d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f7e8103418741398872d59d16e0e919, user_id=d10c68b7fd4446feb759832913fea3b3, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:08.694234 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9b7d78-8a1b-4eb9-b356-ec348603e22d admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:08.751693 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 798/1608] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:53:08 2022] PUT /identity/v3/projects/ba0db0569e094eccb3e8fc578a16fc1e/users/ca0902f6348840729a6ce7a97167da9c/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:08.752633 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514cbd7f-1549-4323-84f4-5df1ab0c1521 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:08.765891 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:08.774913 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 811/1609] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:53:08 2022] PUT /identity/v3/projects/2f7e8103418741398872d59d16e0e919/users/d10c68b7fd4446feb759832913fea3b3/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:08.776234 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cdf5d1-9112-44e5-86f5-838ed466bcb9 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:08.776519 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cdf5d1-9112-44e5-86f5-838ed466bcb9 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:08.776792 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cdf5d1-9112-44e5-86f5-838ed466bcb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:08.807505 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-e0cdf5d1-9112-44e5-86f5-838ed466bcb9 None None] MFA Rules not processed for user `d1aa30210cda4b6a9c457ba56f3a9e04`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:08.808988 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-e0cdf5d1-9112-44e5-86f5-838ed466bcb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:08.820517 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514cbd7f-1549-4323-84f4-5df1ab0c1521 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:08.882143 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f79e02f-852b-40c7-910e-3fc0b3825e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:08.882690 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 812/1610] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:08 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:08.883843 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f79e02f-852b-40c7-910e-3fc0b3825e23 admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:08.884146 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f79e02f-852b-40c7-910e-3fc0b3825e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:08.884426 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f79e02f-852b-40c7-910e-3fc0b3825e23 admin admin] PATH_INFO: `/v3/projects/08576395a1f1437385d2f24b482572a6/users/ef9ff560b7044ea2a18b3c3f8295937e/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:08.884609 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa517195-39a0-4652-9349-279e7914ada1 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:08.884880 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa517195-39a0-4652-9349-279e7914ada1 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:08.885143 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa517195-39a0-4652-9349-279e7914ada1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:08.899776 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f79e02f-852b-40c7-910e-3fc0b3825e23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08576395a1f1437385d2f24b482572a6, user_id=ef9ff560b7044ea2a18b3c3f8295937e, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:08.902432 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f79e02f-852b-40c7-910e-3fc0b3825e23 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:08.903506 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-aa517195-39a0-4652-9349-279e7914ada1 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:08.904723 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-aa517195-39a0-4652-9349-279e7914ada1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:08.912559 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 813/1611] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:08.913478 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa517195-39a0-4652-9349-279e7914ada1 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:08.923433 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa517195-39a0-4652-9349-279e7914ada1 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:08.928182 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a240011-0c10-4e48-9ad0-e9f8d3394e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:08.929030 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a240011-0c10-4e48-9ad0-e9f8d3394e01 admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:08.929312 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a240011-0c10-4e48-9ad0-e9f8d3394e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:08.929583 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a240011-0c10-4e48-9ad0-e9f8d3394e01 admin admin] PATH_INFO: `/v3/projects/e714c93394014ff8875007c80d1d50f3/users/b918c93337964839b3a2abb771e61079/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:08.931871 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:08.944265 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a240011-0c10-4e48-9ad0-e9f8d3394e01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e714c93394014ff8875007c80d1d50f3, user_id=b918c93337964839b3a2abb771e61079, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:08.950505 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a240011-0c10-4e48-9ad0-e9f8d3394e01 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:09.007767 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 799/1612] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:53:08 2022] PUT /identity/v3/projects/08576395a1f1437385d2f24b482572a6/users/ef9ff560b7044ea2a18b3c3f8295937e/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:09.009140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f79e02f-852b-40c7-910e-3fc0b3825e23 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:09.032781 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 814/1613] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:53:08 2022] PUT /identity/v3/projects/e714c93394014ff8875007c80d1d50f3/users/b918c93337964839b3a2abb771e61079/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:09.033724 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a240011-0c10-4e48-9ad0-e9f8d3394e01 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:09.072515 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f79e02f-852b-40c7-910e-3fc0b3825e23 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:09.097901 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a240011-0c10-4e48-9ad0-e9f8d3394e01 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:09.102528 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637c93d9-7892-4680-931b-54572ede2d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:09.103488 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c93d9-7892-4680-931b-54572ede2d92 admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:09.103807 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c93d9-7892-4680-931b-54572ede2d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:09.104077 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c93d9-7892-4680-931b-54572ede2d92 admin admin] PATH_INFO: `/v3/projects/7ee94dac76504026acfa65efdce67cd5/users/b770572014374cfca1053bc07259a30e/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:09.120141 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637c93d9-7892-4680-931b-54572ede2d92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ee94dac76504026acfa65efdce67cd5, user_id=b770572014374cfca1053bc07259a30e, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:09.121422 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:09.123101 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637c93d9-7892-4680-931b-54572ede2d92 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:09.131288 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca239c-f17a-4db8-9317-0a06e1fadbc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:09.132197 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca239c-f17a-4db8-9317-0a06e1fadbc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:09.132567 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca239c-f17a-4db8-9317-0a06e1fadbc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:09.132858 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca239c-f17a-4db8-9317-0a06e1fadbc5 admin admin] PATH_INFO: `/v3/projects/185713cd153d4e9abe3afeabe9d9c723/users/34979454a950418896e803d67b7c66d4/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:09.179106 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca239c-f17a-4db8-9317-0a06e1fadbc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=185713cd153d4e9abe3afeabe9d9c723, user_id=34979454a950418896e803d67b7c66d4, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:09.182301 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca239c-f17a-4db8-9317-0a06e1fadbc5 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:09.191919 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 815/1614] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:53:09 2022] PUT /identity/v3/projects/7ee94dac76504026acfa65efdce67cd5/users/b770572014374cfca1053bc07259a30e/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:09.199008 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23589691-c4f2-4f25-8c5f-15829b6a410f None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:09.199008 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23589691-c4f2-4f25-8c5f-15829b6a410f None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:09.199008 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23589691-c4f2-4f25-8c5f-15829b6a410f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:09.211479 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-23589691-c4f2-4f25-8c5f-15829b6a410f None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:09.212682 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-23589691-c4f2-4f25-8c5f-15829b6a410f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:09.243917 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 800/1615] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:53:09 2022] PUT /identity/v3/projects/185713cd153d4e9abe3afeabe9d9c723/users/34979454a950418896e803d67b7c66d4/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:09.246008 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96dd0a6-f729-4ee0-9ebf-3ce3d184ac5d None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:09.246364 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96dd0a6-f729-4ee0-9ebf-3ce3d184ac5d None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:09.246646 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96dd0a6-f729-4ee0-9ebf-3ce3d184ac5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:09.264143 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-f96dd0a6-f729-4ee0-9ebf-3ce3d184ac5d None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:09.265339 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-f96dd0a6-f729-4ee0-9ebf-3ce3d184ac5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:09.273385 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 816/1616] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:09.274230 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23589691-c4f2-4f25-8c5f-15829b6a410f None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:09.283801 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23589691-c4f2-4f25-8c5f-15829b6a410f None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:09.326166 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 801/1617] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:09.327162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96dd0a6-f729-4ee0-9ebf-3ce3d184ac5d None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:09.341997 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2129e54-19a1-4d1f-a46d-327836ab8c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:09.342694 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2129e54-19a1-4d1f-a46d-327836ab8c0a admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:09.343007 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2129e54-19a1-4d1f-a46d-327836ab8c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:09.343286 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2129e54-19a1-4d1f-a46d-327836ab8c0a admin admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:09.344271 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2129e54-19a1-4d1f-a46d-327836ab8c0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:09.346198 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2129e54-19a1-4d1f-a46d-327836ab8c0a admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:09.356584 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:09.386524 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96dd0a6-f729-4ee0-9ebf-3ce3d184ac5d None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:09.390162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a9cb4d-da25-4ce0-8228-8cac7ec4edf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:09.390838 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a9cb4d-da25-4ce0-8228-8cac7ec4edf6 service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:09.391125 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a9cb4d-da25-4ce0-8228-8cac7ec4edf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:09.391402 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a9cb4d-da25-4ce0-8228-8cac7ec4edf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:09.400701 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a9cb4d-da25-4ce0-8228-8cac7ec4edf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:09.403435 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a9cb4d-da25-4ce0-8228-8cac7ec4edf6 service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:09.411739 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 802/1618] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:53:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:53:09.412921 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f4127a-15b9-4dfc-aedd-02c6692c8edd None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:09.413213 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f4127a-15b9-4dfc-aedd-02c6692c8edd None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:09.413496 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f4127a-15b9-4dfc-aedd-02c6692c8edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:09.431595 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-53f4127a-15b9-4dfc-aedd-02c6692c8edd None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:09.432561 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 817/1619] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:53:09 2022] POST /identity/v3/projects => generated 373 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:53:09.432963 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-53f4127a-15b9-4dfc-aedd-02c6692c8edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:09.433332 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2129e54-19a1-4d1f-a46d-327836ab8c0a admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:09.500518 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2129e54-19a1-4d1f-a46d-327836ab8c0a admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:09.501124 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 803/1620] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:09.502668 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04701038-7d3b-4e37-bcd8-455aa0cb2f80 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:09.503134 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04701038-7d3b-4e37-bcd8-455aa0cb2f80 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:09.503466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04701038-7d3b-4e37-bcd8-455aa0cb2f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:09.509233 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0bb8b2-cc8a-49a9-8156-5cf0a2ceffb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:09.510521 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0bb8b2-cc8a-49a9-8156-5cf0a2ceffb5 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:09.510828 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0bb8b2-cc8a-49a9-8156-5cf0a2ceffb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:09.511105 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0bb8b2-cc8a-49a9-8156-5cf0a2ceffb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:09.514681 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0bb8b2-cc8a-49a9-8156-5cf0a2ceffb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:09.514681 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0bb8b2-cc8a-49a9-8156-5cf0a2ceffb5 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:09.525331 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 818/1621] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:53:09 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:53:09.526379 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0bb8b2-cc8a-49a9-8156-5cf0a2ceffb5 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:09.532089 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-04701038-7d3b-4e37-bcd8-455aa0cb2f80 None None] MFA Rules not processed for user `b770572014374cfca1053bc07259a30e`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:09.533302 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-04701038-7d3b-4e37-bcd8-455aa0cb2f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:09.538506 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:09.544993 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0bb8b2-cc8a-49a9-8156-5cf0a2ceffb5 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:09.552038 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313a7bf1-cda4-4424-8e15-7b785fd974a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:09.552773 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a7bf1-cda4-4424-8e15-7b785fd974a3 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:09.553060 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a7bf1-cda4-4424-8e15-7b785fd974a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:09.553339 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a7bf1-cda4-4424-8e15-7b785fd974a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:09.554071 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313a7bf1-cda4-4424-8e15-7b785fd974a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:09.556163 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313a7bf1-cda4-4424-8e15-7b785fd974a3 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:09.572044 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 819/1622] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:53:09 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:53:09.572044 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313a7bf1-cda4-4424-8e15-7b785fd974a3 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:09.614032 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 804/1623] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:09.614700 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04701038-7d3b-4e37-bcd8-455aa0cb2f80 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:09.629609 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313a7bf1-cda4-4424-8e15-7b785fd974a3 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:09.642667 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e72c19-22ea-46e1-891a-d1eda67f3955 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:09.643148 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e72c19-22ea-46e1-891a-d1eda67f3955 service neutron] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:09.643741 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e72c19-22ea-46e1-891a-d1eda67f3955 service neutron] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:09.644538 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e72c19-22ea-46e1-891a-d1eda67f3955 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:09.661189 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e72c19-22ea-46e1-891a-d1eda67f3955 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:09.661189 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e72c19-22ea-46e1-891a-d1eda67f3955 service neutron] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:09.670280 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 820/1624] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:53:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:53:09.671465 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e72c19-22ea-46e1-891a-d1eda67f3955 service neutron] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:09.679562 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04701038-7d3b-4e37-bcd8-455aa0cb2f80 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:09.683463 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e72c19-22ea-46e1-891a-d1eda67f3955 service neutron] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:09.683778 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9082f9-3edb-4787-9962-2636fa0755aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:09.684429 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9082f9-3edb-4787-9962-2636fa0755aa service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:09.684717 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9082f9-3edb-4787-9962-2636fa0755aa service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:09.684995 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9082f9-3edb-4787-9962-2636fa0755aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:09.688574 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6375a54-1863-459d-b38b-965c5aa223b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:09.689496 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6375a54-1863-459d-b38b-965c5aa223b0 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:09.689999 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6375a54-1863-459d-b38b-965c5aa223b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:09.690491 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6375a54-1863-459d-b38b-965c5aa223b0 admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:09.691428 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6375a54-1863-459d-b38b-965c5aa223b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:09.693860 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6375a54-1863-459d-b38b-965c5aa223b0 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:09.697581 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9082f9-3edb-4787-9962-2636fa0755aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:09.699401 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9082f9-3edb-4787-9962-2636fa0755aa service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:09.710957 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 805/1625] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:53:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:53:09.712049 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9082f9-3edb-4787-9962-2636fa0755aa service neutron] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:09.720564 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9082f9-3edb-4787-9962-2636fa0755aa service neutron] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:09.724400 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca47cdd-ba6e-48b3-8c09-33f505322605 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:09.725043 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca47cdd-ba6e-48b3-8c09-33f505322605 service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:09.725332 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca47cdd-ba6e-48b3-8c09-33f505322605 service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:09.725609 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca47cdd-ba6e-48b3-8c09-33f505322605 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:09.736743 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca47cdd-ba6e-48b3-8c09-33f505322605 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:09.764097 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca47cdd-ba6e-48b3-8c09-33f505322605 service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:09.764097 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 806/1626] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:53:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:53:09.764097 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca47cdd-ba6e-48b3-8c09-33f505322605 service neutron] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:09.764097 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca47cdd-ba6e-48b3-8c09-33f505322605 service neutron] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:09.766464 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3417ab-8848-489d-9c16-63994975f1ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:09.766464 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3417ab-8848-489d-9c16-63994975f1ce admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:09.766464 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3417ab-8848-489d-9c16-63994975f1ce admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:09.766464 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3417ab-8848-489d-9c16-63994975f1ce admin admin] PATH_INFO: `/v3/projects/e714c93394014ff8875007c80d1d50f3/users/b918c93337964839b3a2abb771e61079/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:09.769355 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3417ab-8848-489d-9c16-63994975f1ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=e714c93394014ff8875007c80d1d50f3, user_id=b918c93337964839b3a2abb771e61079, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:09.771263 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3417ab-8848-489d-9c16-63994975f1ce admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:09.782430 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 807/1627] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:53:09 2022] PUT /identity/v3/projects/e714c93394014ff8875007c80d1d50f3/users/b918c93337964839b3a2abb771e61079/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:09.790295 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3417ab-8848-489d-9c16-63994975f1ce admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:09.790439 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 821/1628] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:53:09 2022] POST /identity/v3/users => generated 293 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:53:09.790439 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dbea79-914b-456b-843a-477637f6e9f6 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:09.790439 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dbea79-914b-456b-843a-477637f6e9f6 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:09.790439 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dbea79-914b-456b-843a-477637f6e9f6 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:09.792328 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 822/1629] 10.222.0.52 () {54 vars in 962 bytes} [Thu Jul 21 22:53:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:53:09.794899 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be751b-6e9a-43e0-82ce-c0bd96848a49 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:09.795464 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be751b-6e9a-43e0-82ce-c0bd96848a49 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:09.795931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be751b-6e9a-43e0-82ce-c0bd96848a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:09.829520 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-30be751b-6e9a-43e0-82ce-c0bd96848a49 None None] MFA Rules not processed for user `b918c93337964839b3a2abb771e61079`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:09.830852 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-30be751b-6e9a-43e0-82ce-c0bd96848a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:09.859035 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3417ab-8848-489d-9c16-63994975f1ce admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:09.863750 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386cbb02-1f45-4761-b129-cdd338e3e6cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:09.864637 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386cbb02-1f45-4761-b129-cdd338e3e6cb admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:09.864921 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386cbb02-1f45-4761-b129-cdd338e3e6cb admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:09.865180 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386cbb02-1f45-4761-b129-cdd338e3e6cb admin admin] PATH_INFO: `/v3/projects/185713cd153d4e9abe3afeabe9d9c723/users/34979454a950418896e803d67b7c66d4/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:09.867376 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386cbb02-1f45-4761-b129-cdd338e3e6cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=185713cd153d4e9abe3afeabe9d9c723, user_id=34979454a950418896e803d67b7c66d4, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:09.870509 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386cbb02-1f45-4761-b129-cdd338e3e6cb admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:09.886064 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 808/1630] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:53:09 2022] PUT /identity/v3/projects/185713cd153d4e9abe3afeabe9d9c723/users/34979454a950418896e803d67b7c66d4/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:09.887562 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386cbb02-1f45-4761-b129-cdd338e3e6cb admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:09.899570 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:09.914970 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:09.921696 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 823/1631] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:09 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:09.922999 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ae0f5-9433-4d01-8d0d-3f227185ca16 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:09.923298 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ae0f5-9433-4d01-8d0d-3f227185ca16 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:09.923583 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ae0f5-9433-4d01-8d0d-3f227185ca16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:09.967806 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-c46ae0f5-9433-4d01-8d0d-3f227185ca16 None None] MFA Rules not processed for user `6bfdb655a2a44b33ab724daaf73a294b`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:09.969547 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386cbb02-1f45-4761-b129-cdd338e3e6cb admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:09.969694 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-c46ae0f5-9433-4d01-8d0d-3f227185ca16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:09.975054 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d708583b-3689-45dd-b6ae-4e54f9e3d751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:09.976088 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708583b-3689-45dd-b6ae-4e54f9e3d751 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:09.976207 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708583b-3689-45dd-b6ae-4e54f9e3d751 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:09.977046 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708583b-3689-45dd-b6ae-4e54f9e3d751 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:09.978299 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d708583b-3689-45dd-b6ae-4e54f9e3d751 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:09.980131 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d708583b-3689-45dd-b6ae-4e54f9e3d751 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:09.994231 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 809/1632] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:53:09 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:53:09.996172 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a8a1c-45b3-4b0e-8d1a-ac79a55eb9a2 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:09.997139 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a8a1c-45b3-4b0e-8d1a-ac79a55eb9a2 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:09.997845 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a8a1c-45b3-4b0e-8d1a-ac79a55eb9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:10.028522 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-c44a8a1c-45b3-4b0e-8d1a-ac79a55eb9a2 None None] MFA Rules not processed for user `34979454a950418896e803d67b7c66d4`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:10.029745 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-c44a8a1c-45b3-4b0e-8d1a-ac79a55eb9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:10.045377 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 824/1633] 10.222.0.52 () {58 vars in 1059 bytes} [Thu Jul 21 22:53:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:10.046162 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ae0f5-9433-4d01-8d0d-3f227185ca16 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:10.109658 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 810/1634] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:09 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:10.110995 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a8a1c-45b3-4b0e-8d1a-ac79a55eb9a2 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:10.118994 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ae0f5-9433-4d01-8d0d-3f227185ca16 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:10.124889 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a8a1c-45b3-4b0e-8d1a-ac79a55eb9a2 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:10.130869 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b960d7-f00e-417f-b9d6-04c6d0d71414 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:10.130869 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b960d7-f00e-417f-b9d6-04c6d0d71414 service cinder] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:10.130869 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b960d7-f00e-417f-b9d6-04c6d0d71414 service cinder] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:10.130869 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b960d7-f00e-417f-b9d6-04c6d0d71414 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:10.131343 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef742520-fd0e-4c3b-b0b0-e689df5eeb3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:10.132285 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef742520-fd0e-4c3b-b0b0-e689df5eeb3e admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:10.132577 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef742520-fd0e-4c3b-b0b0-e689df5eeb3e admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:10.132849 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef742520-fd0e-4c3b-b0b0-e689df5eeb3e admin admin] PATH_INFO: `/v3/projects/cbdcb370ad4041aeaa7ce1f28ac041ed/users/f547635351a04a62a97f196ec35e4d0b/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:10.148791 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef742520-fd0e-4c3b-b0b0-e689df5eeb3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbdcb370ad4041aeaa7ce1f28ac041ed, user_id=f547635351a04a62a97f196ec35e4d0b, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:10.151562 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef742520-fd0e-4c3b-b0b0-e689df5eeb3e admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:10.196740 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b960d7-f00e-417f-b9d6-04c6d0d71414 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:10.198171 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b960d7-f00e-417f-b9d6-04c6d0d71414 service cinder] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:10.209364 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 825/1635] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:53:10 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:53:10.210216 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b960d7-f00e-417f-b9d6-04c6d0d71414 service cinder] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:10.222952 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b960d7-f00e-417f-b9d6-04c6d0d71414 service cinder] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:10.230326 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff10c394-c5e7-4f7a-a20d-5303e7455e0e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bfdb655a2a44b33ab724daaf73a294b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:10.231046 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff10c394-c5e7-4f7a-a20d-5303e7455e0e service nova] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:10.231457 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff10c394-c5e7-4f7a-a20d-5303e7455e0e service nova] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:10.231732 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff10c394-c5e7-4f7a-a20d-5303e7455e0e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:10.248301 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:10.249821 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 811/1636] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:53:10 2022] PUT /identity/v3/projects/cbdcb370ad4041aeaa7ce1f28ac041ed/users/f547635351a04a62a97f196ec35e4d0b/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:10.251329 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef742520-fd0e-4c3b-b0b0-e689df5eeb3e admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:10.307698 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff10c394-c5e7-4f7a-a20d-5303e7455e0e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:10.309698 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff10c394-c5e7-4f7a-a20d-5303e7455e0e service nova] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:10.337795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 826/1637] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:53:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:53:10.339010 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff10c394-c5e7-4f7a-a20d-5303e7455e0e service nova] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:10.361739 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff10c394-c5e7-4f7a-a20d-5303e7455e0e service nova] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:10.368673 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef742520-fd0e-4c3b-b0b0-e689df5eeb3e admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:10.376024 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29af5bdc-6a62-4b01-b981-fdd66cca2fdc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:10.376697 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af5bdc-6a62-4b01-b981-fdd66cca2fdc service cinder] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:10.377030 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af5bdc-6a62-4b01-b981-fdd66cca2fdc service cinder] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:10.377319 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af5bdc-6a62-4b01-b981-fdd66cca2fdc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:10.416037 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:10.451989 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc902c5-2531-4b1f-9edb-de933413d721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:10.452340 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29af5bdc-6a62-4b01-b981-fdd66cca2fdc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:10.452839 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc902c5-2531-4b1f-9edb-de933413d721 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:10.453134 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc902c5-2531-4b1f-9edb-de933413d721 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:10.453417 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc902c5-2531-4b1f-9edb-de933413d721 admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:10.454502 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29af5bdc-6a62-4b01-b981-fdd66cca2fdc service cinder] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:10.454644 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc902c5-2531-4b1f-9edb-de933413d721 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:10.455744 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc902c5-2531-4b1f-9edb-de933413d721 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:10.468017 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 812/1638] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:53:10 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:53:10.471992 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29af5bdc-6a62-4b01-b981-fdd66cca2fdc service cinder] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:10.472157 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 827/1639] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:53:10 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:53:10.498985 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29af5bdc-6a62-4b01-b981-fdd66cca2fdc service cinder] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:10.505310 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd798d6b-d0d7-4e5b-9252-c01285f9a728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:10.506554 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd798d6b-d0d7-4e5b-9252-c01285f9a728 admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:10.507014 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd798d6b-d0d7-4e5b-9252-c01285f9a728 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:10.507499 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd798d6b-d0d7-4e5b-9252-c01285f9a728 admin admin] PATH_INFO: `/v3/projects/cbdcb370ad4041aeaa7ce1f28ac041ed/users/f547635351a04a62a97f196ec35e4d0b/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:10.511384 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd798d6b-d0d7-4e5b-9252-c01285f9a728 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbdcb370ad4041aeaa7ce1f28ac041ed, user_id=f547635351a04a62a97f196ec35e4d0b, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:10.514735 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd798d6b-d0d7-4e5b-9252-c01285f9a728 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:10.530842 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 813/1640] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:53:10 2022] PUT /identity/v3/projects/cbdcb370ad4041aeaa7ce1f28ac041ed/users/f547635351a04a62a97f196ec35e4d0b/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:10.536358 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc902c5-2531-4b1f-9edb-de933413d721 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:10.540614 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28531721-f6b9-4e0c-b676-cb5a304fa323 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:10.543679 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28531721-f6b9-4e0c-b676-cb5a304fa323 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:10.543679 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28531721-f6b9-4e0c-b676-cb5a304fa323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:10.551662 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc902c5-2531-4b1f-9edb-de933413d721 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:10.575579 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-28531721-f6b9-4e0c-b676-cb5a304fa323 None None] MFA Rules not processed for user `f547635351a04a62a97f196ec35e4d0b`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:10.577080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-28531721-f6b9-4e0c-b676-cb5a304fa323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:10.657616 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb6b3a7-fc5b-4720-9d65-a2e73f8c53b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:10.658478 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6b3a7-fc5b-4720-9d65-a2e73f8c53b8 service cinder] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:10.659312 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6b3a7-fc5b-4720-9d65-a2e73f8c53b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:10.671020 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6b3a7-fc5b-4720-9d65-a2e73f8c53b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:10.710913 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 814/1641] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:10 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:10.714315 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f811ae2-db08-4a91-a18c-aecb44d8d5ed None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:10.715682 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f811ae2-db08-4a91-a18c-aecb44d8d5ed None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:10.719489 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f811ae2-db08-4a91-a18c-aecb44d8d5ed None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:10.740838 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 815/1642] 10.222.0.52 () {54 vars in 912 bytes} [Thu Jul 21 22:53:10 2022] GET /identity => generated 270 bytes in 29 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:53:10.742249 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc4238-974a-42ce-ad74-c287b5423080 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:10.744039 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc4238-974a-42ce-ad74-c287b5423080 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:10.744039 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc4238-974a-42ce-ad74-c287b5423080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:10.767071 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-abdc4238-974a-42ce-ad74-c287b5423080 None None] MFA Rules not processed for user `6bfdb655a2a44b33ab724daaf73a294b`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:10.768130 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb6b3a7-fc5b-4720-9d65-a2e73f8c53b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:10.768363 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-abdc4238-974a-42ce-ad74-c287b5423080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:10.769638 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb6b3a7-fc5b-4720-9d65-a2e73f8c53b8 service cinder] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:10.780635 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 828/1643] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:53:10 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:53:10.783178 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb6b3a7-fc5b-4720-9d65-a2e73f8c53b8 service cinder] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:10.789767 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:10.793737 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb6b3a7-fc5b-4720-9d65-a2e73f8c53b8 service cinder] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:10.798752 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acadbfa1-7a3a-45f7-b9ca-86f924cacd7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:10.799620 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acadbfa1-7a3a-45f7-b9ca-86f924cacd7d service cinder] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:10.799975 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acadbfa1-7a3a-45f7-b9ca-86f924cacd7d service cinder] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:10.800350 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acadbfa1-7a3a-45f7-b9ca-86f924cacd7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:10.814390 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acadbfa1-7a3a-45f7-b9ca-86f924cacd7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:10.819224 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acadbfa1-7a3a-45f7-b9ca-86f924cacd7d service cinder] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:10.828868 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 829/1644] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:53:10 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:53:10.850955 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 816/1645] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jul 21 22:53:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:11.085954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acadbfa1-7a3a-45f7-b9ca-86f924cacd7d service cinder] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:11.104839 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acadbfa1-7a3a-45f7-b9ca-86f924cacd7d service cinder] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:11.109403 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bfe7bb-0da9-4992-b4f5-029acfe4df19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:11.110060 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bfe7bb-0da9-4992-b4f5-029acfe4df19 service cinder] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:11.110406 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bfe7bb-0da9-4992-b4f5-029acfe4df19 service cinder] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:11.110639 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bfe7bb-0da9-4992-b4f5-029acfe4df19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:11.123601 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bfe7bb-0da9-4992-b4f5-029acfe4df19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:11.127745 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bfe7bb-0da9-4992-b4f5-029acfe4df19 service cinder] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:11.137844 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 830/1646] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:53:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:53:11.740265 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdc4238-974a-42ce-ad74-c287b5423080 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:11.750156 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdc4238-974a-42ce-ad74-c287b5423080 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:11.753895 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56855ae4-886f-41da-9c4c-22706358f3b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:11.754546 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56855ae4-886f-41da-9c4c-22706358f3b0 service cinder] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:11.754865 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56855ae4-886f-41da-9c4c-22706358f3b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:11.755150 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56855ae4-886f-41da-9c4c-22706358f3b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:11.795975 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:11.850595 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56855ae4-886f-41da-9c4c-22706358f3b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:11.853333 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56855ae4-886f-41da-9c4c-22706358f3b0 service cinder] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:11.862787 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 817/1647] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:53:11 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:53:12.642462 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003708dd-6660-4c3a-847e-47f4a74a5f21 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:12.644441 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003708dd-6660-4c3a-847e-47f4a74a5f21 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:12.645489 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003708dd-6660-4c3a-847e-47f4a74a5f21 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:12.648447 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 831/1648] 10.222.0.52 () {54 vars in 962 bytes} [Thu Jul 21 22:53:12 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:53:12.671750 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1f221-d92d-4859-9d29-138f8706d2e9 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:12.672658 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1f221-d92d-4859-9d29-138f8706d2e9 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:12.673802 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1f221-d92d-4859-9d29-138f8706d2e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:12.711934 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-afb1f221-d92d-4859-9d29-138f8706d2e9 None None] MFA Rules not processed for user `6bfdb655a2a44b33ab724daaf73a294b`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:12.713660 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-afb1f221-d92d-4859-9d29-138f8706d2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:12.731613 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 818/1649] 10.222.0.52 () {58 vars in 1059 bytes} [Thu Jul 21 22:53:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:12.737319 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5921615-60b6-4a00-98e8-587d1e2cc3aa None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:12.738093 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5921615-60b6-4a00-98e8-587d1e2cc3aa None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:12.738731 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5921615-60b6-4a00-98e8-587d1e2cc3aa None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:12.743268 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 832/1650] 10.222.0.52 () {54 vars in 912 bytes} [Thu Jul 21 22:53:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:53:12.750274 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3108350-0ca6-43e2-9ed6-5fb1e05f1a02 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:12.750274 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3108350-0ca6-43e2-9ed6-5fb1e05f1a02 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:12.750274 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3108350-0ca6-43e2-9ed6-5fb1e05f1a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:12.791685 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-e3108350-0ca6-43e2-9ed6-5fb1e05f1a02 None None] MFA Rules not processed for user `6bfdb655a2a44b33ab724daaf73a294b`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:12.799930 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-e3108350-0ca6-43e2-9ed6-5fb1e05f1a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:12.826151 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 819/1651] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jul 21 22:53:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:12.887081 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5921615-60b6-4a00-98e8-587d1e2cc3aa None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:12.902101 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5921615-60b6-4a00-98e8-587d1e2cc3aa None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:12.907274 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b62700-d68c-4eee-9a38-d2dd51b196ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:12.908122 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b62700-d68c-4eee-9a38-d2dd51b196ab service cinder] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:12.908593 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b62700-d68c-4eee-9a38-d2dd51b196ab service cinder] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:12.909085 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b62700-d68c-4eee-9a38-d2dd51b196ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:12.955976 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b62700-d68c-4eee-9a38-d2dd51b196ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:12.958155 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b62700-d68c-4eee-9a38-d2dd51b196ab service cinder] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:13.003195 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 833/1652] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:53:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:53:16.641808 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3108350-0ca6-43e2-9ed6-5fb1e05f1a02 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:16.818717 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3108350-0ca6-43e2-9ed6-5fb1e05f1a02 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:16.828434 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8d8835-517b-4bf2-9ac9-a19f81646162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:16.831508 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8d8835-517b-4bf2-9ac9-a19f81646162 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:16.831818 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8d8835-517b-4bf2-9ac9-a19f81646162 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:16.832107 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8d8835-517b-4bf2-9ac9-a19f81646162 admin admin] PATH_INFO: `/v3/users/d9bcda22585e4e6ea8bf19b162406566` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:16.837790 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8d8835-517b-4bf2-9ac9-a19f81646162 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9bcda22585e4e6ea8bf19b162406566)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:16.838736 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: WARNING py.warnings [None req-ee8d8835-517b-4bf2-9ac9-a19f81646162 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 22:53:16.838736 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: warnings.warn(msg) Jul 21 22:53:16.838736 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: Jul 21 22:53:16.840090 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8d8835-517b-4bf2-9ac9-a19f81646162 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:17.208212 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-ee8d8835-517b-4bf2-9ac9-a19f81646162 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9bcda22585e4e6ea8bf19b162406566'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:17.236309 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-ee8d8835-517b-4bf2-9ac9-a19f81646162 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9bcda22585e4e6ea8bf19b162406566'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:17.239298 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-ee8d8835-517b-4bf2-9ac9-a19f81646162 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9bcda22585e4e6ea8bf19b162406566'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:17.332382 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-ee8d8835-517b-4bf2-9ac9-a19f81646162 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9bcda22585e4e6ea8bf19b162406566'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:17.430989 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-ee8d8835-517b-4bf2-9ac9-a19f81646162 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9bcda22585e4e6ea8bf19b162406566'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:17.440410 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 820/1653] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:53:16 2022] DELETE /identity/v3/users/d9bcda22585e4e6ea8bf19b162406566 => generated 0 bytes in 799 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:17.444674 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365262b2-3fdf-446c-9c7b-4d9402aae550 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:17.445603 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365262b2-3fdf-446c-9c7b-4d9402aae550 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:17.446029 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365262b2-3fdf-446c-9c7b-4d9402aae550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:17.481607 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-365262b2-3fdf-446c-9c7b-4d9402aae550 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:17.484066 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-365262b2-3fdf-446c-9c7b-4d9402aae550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:17.636990 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 834/1654] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:17.661984 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8d8835-517b-4bf2-9ac9-a19f81646162 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:17.664922 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-ee8d8835-517b-4bf2-9ac9-a19f81646162 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:17.709840 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:17.769862 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8d8835-517b-4bf2-9ac9-a19f81646162 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:17.779458 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3fd609-106e-49ef-993f-79963a86eb07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:17.780247 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3fd609-106e-49ef-993f-79963a86eb07 service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:17.780604 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3fd609-106e-49ef-993f-79963a86eb07 service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:17.781010 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3fd609-106e-49ef-993f-79963a86eb07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:17.803145 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365262b2-3fdf-446c-9c7b-4d9402aae550 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:17.805989 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-365262b2-3fdf-446c-9c7b-4d9402aae550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:17.808748 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3fd609-106e-49ef-993f-79963a86eb07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:17.810570 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3fd609-106e-49ef-993f-79963a86eb07 service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:17.827746 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365262b2-3fdf-446c-9c7b-4d9402aae550 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:17.841606 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 821/1655] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:53:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:53:17.843208 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d73683-9d6b-4a64-b04e-70428ce2ece6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:17.844377 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d73683-9d6b-4a64-b04e-70428ce2ece6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:17.844661 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d73683-9d6b-4a64-b04e-70428ce2ece6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:17.844941 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d73683-9d6b-4a64-b04e-70428ce2ece6 admin admin] PATH_INFO: `/v3/users/57db3b9e6fe642c9ad51f10dfc62c59f` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:17.847358 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d73683-9d6b-4a64-b04e-70428ce2ece6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57db3b9e6fe642c9ad51f10dfc62c59f)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:17.853437 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d73683-9d6b-4a64-b04e-70428ce2ece6 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:18.228395 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-28d73683-9d6b-4a64-b04e-70428ce2ece6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57db3b9e6fe642c9ad51f10dfc62c59f'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:18.256803 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-28d73683-9d6b-4a64-b04e-70428ce2ece6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57db3b9e6fe642c9ad51f10dfc62c59f'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:18.260384 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-28d73683-9d6b-4a64-b04e-70428ce2ece6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57db3b9e6fe642c9ad51f10dfc62c59f'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:18.301541 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-28d73683-9d6b-4a64-b04e-70428ce2ece6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57db3b9e6fe642c9ad51f10dfc62c59f'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:18.302496 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-28d73683-9d6b-4a64-b04e-70428ce2ece6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57db3b9e6fe642c9ad51f10dfc62c59f'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:18.393349 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 835/1656] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:53:17 2022] DELETE /identity/v3/users/57db3b9e6fe642c9ad51f10dfc62c59f => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:53:18.398580 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f59033-9537-4949-a7e3-2ba1c837d1ed None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:18.401764 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f59033-9537-4949-a7e3-2ba1c837d1ed None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:18.402265 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f59033-9537-4949-a7e3-2ba1c837d1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:18.721582 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-d9f59033-9537-4949-a7e3-2ba1c837d1ed None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:18.721582 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-d9f59033-9537-4949-a7e3-2ba1c837d1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:18.721582 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 822/1657] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:18.751193 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d73683-9d6b-4a64-b04e-70428ce2ece6 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:18.751193 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-28d73683-9d6b-4a64-b04e-70428ce2ece6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:18.858866 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:18.889461 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d73683-9d6b-4a64-b04e-70428ce2ece6 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:18.898732 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7125642c-d5f6-4d94-b032-7086907ee581 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:18.902248 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7125642c-d5f6-4d94-b032-7086907ee581 service neutron] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:18.902560 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7125642c-d5f6-4d94-b032-7086907ee581 service neutron] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:18.906571 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7125642c-d5f6-4d94-b032-7086907ee581 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:18.928142 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7125642c-d5f6-4d94-b032-7086907ee581 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:18.930441 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7125642c-d5f6-4d94-b032-7086907ee581 service neutron] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:18.942228 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 836/1658] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:53:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:53:18.980489 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f59033-9537-4949-a7e3-2ba1c837d1ed None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:18.982713 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-d9f59033-9537-4949-a7e3-2ba1c837d1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:19.005048 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f59033-9537-4949-a7e3-2ba1c837d1ed None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:19.013480 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe20a7f-4661-4de9-8b85-3a85ca153485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:19.015437 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe20a7f-4661-4de9-8b85-3a85ca153485 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:19.016989 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe20a7f-4661-4de9-8b85-3a85ca153485 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:19.017256 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe20a7f-4661-4de9-8b85-3a85ca153485 admin admin] PATH_INFO: `/v3/projects/faadfc4e89d649c1b54da92755178fee` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:19.023851 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe20a7f-4661-4de9-8b85-3a85ca153485 admin admin] RBAC: Authorizing `identity:delete_project(project_id=faadfc4e89d649c1b54da92755178fee)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:19.041520 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe20a7f-4661-4de9-8b85-3a85ca153485 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:19.203136 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-7fe20a7f-4661-4de9-8b85-3a85ca153485 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'faadfc4e89d649c1b54da92755178fee', 'request_id': 'req-7fe20a7f-4661-4de9-8b85-3a85ca153485'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:19.212210 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-7fe20a7f-4661-4de9-8b85-3a85ca153485 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'faadfc4e89d649c1b54da92755178fee', 'request_id': 'req-7fe20a7f-4661-4de9-8b85-3a85ca153485'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:19.325004 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-7fe20a7f-4661-4de9-8b85-3a85ca153485 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'faadfc4e89d649c1b54da92755178fee', 'request_id': 'req-7fe20a7f-4661-4de9-8b85-3a85ca153485'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:19.388622 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-7fe20a7f-4661-4de9-8b85-3a85ca153485 admin admin] The token cache is being invalidate because project faadfc4e89d649c1b54da92755178fee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:53:19.389791 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-7fe20a7f-4661-4de9-8b85-3a85ca153485 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:19.393242 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-7fe20a7f-4661-4de9-8b85-3a85ca153485 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:19.403123 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 823/1659] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:53:18 2022] DELETE /identity/v3/projects/faadfc4e89d649c1b54da92755178fee => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:19.408248 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7125642c-d5f6-4d94-b032-7086907ee581 service neutron] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:19.424133 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7125642c-d5f6-4d94-b032-7086907ee581 service neutron] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:19.557300 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab207d-bf63-4e42-a34c-8bdd026aea72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:19.558218 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab207d-bf63-4e42-a34c-8bdd026aea72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:19.558509 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab207d-bf63-4e42-a34c-8bdd026aea72 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:19.559105 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab207d-bf63-4e42-a34c-8bdd026aea72 admin admin] PATH_INFO: `/v3/users/b918c93337964839b3a2abb771e61079` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:19.564614 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ab207d-bf63-4e42-a34c-8bdd026aea72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b918c93337964839b3a2abb771e61079)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:19.580560 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ab207d-bf63-4e42-a34c-8bdd026aea72 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:19.834010 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-37ab207d-bf63-4e42-a34c-8bdd026aea72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b918c93337964839b3a2abb771e61079'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:19.887737 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-37ab207d-bf63-4e42-a34c-8bdd026aea72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b918c93337964839b3a2abb771e61079'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:19.889656 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-37ab207d-bf63-4e42-a34c-8bdd026aea72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b918c93337964839b3a2abb771e61079'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:19.929362 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-37ab207d-bf63-4e42-a34c-8bdd026aea72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b918c93337964839b3a2abb771e61079'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:19.930556 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-37ab207d-bf63-4e42-a34c-8bdd026aea72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b918c93337964839b3a2abb771e61079'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:20.019703 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 837/1660] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:53:19 2022] DELETE /identity/v3/users/b918c93337964839b3a2abb771e61079 => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:20.438798 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff4933-4c06-4971-97f6-31146468ded5 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:20.444779 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff4933-4c06-4971-97f6-31146468ded5 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:20.444779 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff4933-4c06-4971-97f6-31146468ded5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:20.514217 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-84ff4933-4c06-4971-97f6-31146468ded5 None None] MFA Rules not processed for user `ca0902f6348840729a6ce7a97167da9c`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:20.518141 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-84ff4933-4c06-4971-97f6-31146468ded5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:20.635715 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:20.658212 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 824/1661] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:20.670479 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a725a6a-ca06-44db-8c4e-b34e8937a01d None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:20.670762 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a725a6a-ca06-44db-8c4e-b34e8937a01d None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:20.671058 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a725a6a-ca06-44db-8c4e-b34e8937a01d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:20.705646 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-4a725a6a-ca06-44db-8c4e-b34e8937a01d None None] MFA Rules not processed for user `ca0902f6348840729a6ce7a97167da9c`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:20.717459 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-4a725a6a-ca06-44db-8c4e-b34e8937a01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:20.738894 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 838/1662] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:20.748313 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ff4933-4c06-4971-97f6-31146468ded5 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:20.754539 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-84ff4933-4c06-4971-97f6-31146468ded5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:20.889222 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ff4933-4c06-4971-97f6-31146468ded5 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:20.896366 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272121d7-196f-4613-9115-77a3b64e65ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bfdb655a2a44b33ab724daaf73a294b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:20.901219 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272121d7-196f-4613-9115-77a3b64e65ad service nova] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:20.901219 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272121d7-196f-4613-9115-77a3b64e65ad service nova] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:20.901219 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272121d7-196f-4613-9115-77a3b64e65ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:20.915340 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272121d7-196f-4613-9115-77a3b64e65ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:20.917048 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272121d7-196f-4613-9115-77a3b64e65ad service nova] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:20.944097 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 825/1663] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:53:20 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:53:21.269161 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a725a6a-ca06-44db-8c4e-b34e8937a01d None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:21.270699 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-4a725a6a-ca06-44db-8c4e-b34e8937a01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:21.358166 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272121d7-196f-4613-9115-77a3b64e65ad service nova] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:21.359854 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-272121d7-196f-4613-9115-77a3b64e65ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:21.428091 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:21.451985 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a725a6a-ca06-44db-8c4e-b34e8937a01d None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:21.457986 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9762f1-e067-4d17-a09e-24b564141876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:21.459915 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9762f1-e067-4d17-a09e-24b564141876 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:21.464081 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9762f1-e067-4d17-a09e-24b564141876 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:21.464409 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9762f1-e067-4d17-a09e-24b564141876 admin admin] PATH_INFO: `/v3/projects/cdb170fd25234969b1ab3e7b828dca86` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:21.466174 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9762f1-e067-4d17-a09e-24b564141876 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdb170fd25234969b1ab3e7b828dca86)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:21.469727 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: WARNING py.warnings [None req-7d9762f1-e067-4d17-a09e-24b564141876 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 22:53:21.469727 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: warnings.warn(msg) Jul 21 22:53:21.469727 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: Jul 21 22:53:21.471502 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9762f1-e067-4d17-a09e-24b564141876 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:21.540377 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272121d7-196f-4613-9115-77a3b64e65ad service nova] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:21.558366 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49a0b79-5315-4271-a24d-393407336cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:21.561973 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49a0b79-5315-4271-a24d-393407336cf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:21.562407 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49a0b79-5315-4271-a24d-393407336cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:21.562745 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49a0b79-5315-4271-a24d-393407336cf4 admin admin] PATH_INFO: `/v3/projects/e714c93394014ff8875007c80d1d50f3` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:21.564752 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49a0b79-5315-4271-a24d-393407336cf4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e714c93394014ff8875007c80d1d50f3)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:21.570364 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49a0b79-5315-4271-a24d-393407336cf4 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:21.715362 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-7d9762f1-e067-4d17-a09e-24b564141876 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdb170fd25234969b1ab3e7b828dca86', 'request_id': 'req-7d9762f1-e067-4d17-a09e-24b564141876'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:21.726271 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-a49a0b79-5315-4271-a24d-393407336cf4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e714c93394014ff8875007c80d1d50f3', 'request_id': 'req-a49a0b79-5315-4271-a24d-393407336cf4'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:21.733583 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-7d9762f1-e067-4d17-a09e-24b564141876 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdb170fd25234969b1ab3e7b828dca86', 'request_id': 'req-7d9762f1-e067-4d17-a09e-24b564141876'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:21.742819 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-a49a0b79-5315-4271-a24d-393407336cf4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e714c93394014ff8875007c80d1d50f3', 'request_id': 'req-a49a0b79-5315-4271-a24d-393407336cf4'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:21.832502 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-7d9762f1-e067-4d17-a09e-24b564141876 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdb170fd25234969b1ab3e7b828dca86', 'request_id': 'req-7d9762f1-e067-4d17-a09e-24b564141876'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:21.880648 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-7d9762f1-e067-4d17-a09e-24b564141876 admin admin] The token cache is being invalidate because project cdb170fd25234969b1ab3e7b828dca86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:53:21.880952 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-7d9762f1-e067-4d17-a09e-24b564141876 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:21.886751 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-7d9762f1-e067-4d17-a09e-24b564141876 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:21.894766 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 839/1664] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:53:21 2022] DELETE /identity/v3/projects/cdb170fd25234969b1ab3e7b828dca86 => generated 0 bytes in 625 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:21.897213 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9762f1-e067-4d17-a09e-24b564141876 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:21.899916 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-7d9762f1-e067-4d17-a09e-24b564141876 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:21.928386 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-a49a0b79-5315-4271-a24d-393407336cf4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e714c93394014ff8875007c80d1d50f3', 'request_id': 'req-a49a0b79-5315-4271-a24d-393407336cf4'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:21.949671 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-a49a0b79-5315-4271-a24d-393407336cf4 admin admin] The token cache is being invalidate because project e714c93394014ff8875007c80d1d50f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:53:21.950232 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-a49a0b79-5315-4271-a24d-393407336cf4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:21.954561 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-a49a0b79-5315-4271-a24d-393407336cf4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:21.961001 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 826/1665] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:53:21 2022] DELETE /identity/v3/projects/e714c93394014ff8875007c80d1d50f3 => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:21.963226 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bacb47-68b8-490f-8e01-e057fa14b9df None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:21.963761 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bacb47-68b8-490f-8e01-e057fa14b9df None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:21.964348 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bacb47-68b8-490f-8e01-e057fa14b9df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:22.012486 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-52bacb47-68b8-490f-8e01-e057fa14b9df None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:22.016189 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-52bacb47-68b8-490f-8e01-e057fa14b9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:22.021653 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9762f1-e067-4d17-a09e-24b564141876 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:22.144150 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ad7da7-b9e5-462f-8cd8-5ac0876d4dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:22.152345 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad7da7-b9e5-462f-8cd8-5ac0876d4dcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:22.152345 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad7da7-b9e5-462f-8cd8-5ac0876d4dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:22.152345 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad7da7-b9e5-462f-8cd8-5ac0876d4dcd admin admin] PATH_INFO: `/v3/users/34979454a950418896e803d67b7c66d4` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:22.152345 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ad7da7-b9e5-462f-8cd8-5ac0876d4dcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=34979454a950418896e803d67b7c66d4)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:22.152345 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ad7da7-b9e5-462f-8cd8-5ac0876d4dcd admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:22.181143 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 827/1666] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:22.188127 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d67b2-7cad-4fa2-b41c-fe55fc138238 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:22.188708 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d67b2-7cad-4fa2-b41c-fe55fc138238 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:22.189115 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d67b2-7cad-4fa2-b41c-fe55fc138238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:22.221489 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-dd6d67b2-7cad-4fa2-b41c-fe55fc138238 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:22.221489 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-dd6d67b2-7cad-4fa2-b41c-fe55fc138238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:22.236136 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 828/1667] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:22.236136 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c55143d-ed36-494f-aa19-bc42a3deb64b None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:22.236136 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c55143d-ed36-494f-aa19-bc42a3deb64b None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:22.236136 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c55143d-ed36-494f-aa19-bc42a3deb64b None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:22.237906 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 829/1668] 10.222.0.52 () {54 vars in 912 bytes} [Thu Jul 21 22:53:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:53:22.238705 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c55143d-ed36-494f-aa19-bc42a3deb64b None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:22.259675 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c55143d-ed36-494f-aa19-bc42a3deb64b None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:22.269561 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1d046-1237-4c29-8389-0d12b608d71b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:22.270527 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1d046-1237-4c29-8389-0d12b608d71b admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:22.271017 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1d046-1237-4c29-8389-0d12b608d71b admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:22.271823 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1d046-1237-4c29-8389-0d12b608d71b admin admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:22.273766 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb1d046-1237-4c29-8389-0d12b608d71b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:53:22.279004 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb1d046-1237-4c29-8389-0d12b608d71b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:22.280936 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb1d046-1237-4c29-8389-0d12b608d71b admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:22.306310 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 830/1669] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 22:53:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:53:22.306310 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14538f9a-27f2-49b7-8e85-4fe1d89d814a None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:22.306310 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14538f9a-27f2-49b7-8e85-4fe1d89d814a None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:22.306310 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14538f9a-27f2-49b7-8e85-4fe1d89d814a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:22.322643 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:22.381161 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-14538f9a-27f2-49b7-8e85-4fe1d89d814a None None] MFA Rules not processed for user `48c3e33f24ff4d66a72f06848d769ac7`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:22.382123 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-14538f9a-27f2-49b7-8e85-4fe1d89d814a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:22.487897 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 831/1670] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jul 21 22:53:22 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:22.489115 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14538f9a-27f2-49b7-8e85-4fe1d89d814a None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:22.514740 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14538f9a-27f2-49b7-8e85-4fe1d89d814a None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:22.517327 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-e7ad7da7-b9e5-462f-8cd8-5ac0876d4dcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34979454a950418896e803d67b7c66d4'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:22.525724 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3481f57f-3e18-44e5-8cce-36a94aa48c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:22.526601 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481f57f-3e18-44e5-8cce-36a94aa48c64 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:22.530048 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481f57f-3e18-44e5-8cce-36a94aa48c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:22.530048 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481f57f-3e18-44e5-8cce-36a94aa48c64 admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:22.530048 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3481f57f-3e18-44e5-8cce-36a94aa48c64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:22.531044 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3481f57f-3e18-44e5-8cce-36a94aa48c64 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:22.539396 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-e7ad7da7-b9e5-462f-8cd8-5ac0876d4dcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34979454a950418896e803d67b7c66d4'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:22.541911 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-e7ad7da7-b9e5-462f-8cd8-5ac0876d4dcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34979454a950418896e803d67b7c66d4'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:22.575786 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-e7ad7da7-b9e5-462f-8cd8-5ac0876d4dcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34979454a950418896e803d67b7c66d4'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:22.578459 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-e7ad7da7-b9e5-462f-8cd8-5ac0876d4dcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34979454a950418896e803d67b7c66d4'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:22.644785 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 832/1671] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:53:22 2022] POST /identity/v3/projects => generated 385 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:53:22.646276 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3481f57f-3e18-44e5-8cce-36a94aa48c64 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:22.650188 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-3481f57f-3e18-44e5-8cce-36a94aa48c64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:22.673024 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 840/1672] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:53:21 2022] DELETE /identity/v3/users/34979454a950418896e803d67b7c66d4 => generated 0 bytes in 777 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:22.674135 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ad7da7-b9e5-462f-8cd8-5ac0876d4dcd admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:22.676393 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-e7ad7da7-b9e5-462f-8cd8-5ac0876d4dcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:22.803158 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3481f57f-3e18-44e5-8cce-36a94aa48c64 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:22.915126 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ad7da7-b9e5-462f-8cd8-5ac0876d4dcd admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:22.924260 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38879068-0afb-47cb-8268-dc362f89270e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:22.924260 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38879068-0afb-47cb-8268-dc362f89270e service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:22.924260 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38879068-0afb-47cb-8268-dc362f89270e service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:22.924260 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38879068-0afb-47cb-8268-dc362f89270e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:22.924633 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-38879068-0afb-47cb-8268-dc362f89270e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:22.926438 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd78e21-d34e-4212-ac4b-2ca86f2d8e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:22.928784 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd78e21-d34e-4212-ac4b-2ca86f2d8e13 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:22.929109 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd78e21-d34e-4212-ac4b-2ca86f2d8e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:22.931108 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd78e21-d34e-4212-ac4b-2ca86f2d8e13 admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:22.931827 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd78e21-d34e-4212-ac4b-2ca86f2d8e13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:22.940250 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd78e21-d34e-4212-ac4b-2ca86f2d8e13 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:22.948058 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38879068-0afb-47cb-8268-dc362f89270e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:22.950887 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38879068-0afb-47cb-8268-dc362f89270e service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:22.967771 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 833/1673] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:53:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:53:23.045238 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 841/1674] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:53:22 2022] POST /identity/v3/users => generated 293 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:53:23.046710 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38879068-0afb-47cb-8268-dc362f89270e service neutron] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:23.065962 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38879068-0afb-47cb-8268-dc362f89270e service neutron] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:23.080045 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2cb862-b8f6-4ad6-b58d-71efbbc830f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:23.083885 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2cb862-b8f6-4ad6-b58d-71efbbc830f8 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:23.084306 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2cb862-b8f6-4ad6-b58d-71efbbc830f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:23.084689 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2cb862-b8f6-4ad6-b58d-71efbbc830f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:23.085836 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2cb862-b8f6-4ad6-b58d-71efbbc830f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:23.089202 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2cb862-b8f6-4ad6-b58d-71efbbc830f8 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:23.127524 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 834/1675] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:53:23 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:53:23.129417 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd78e21-d34e-4212-ac4b-2ca86f2d8e13 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:23.147224 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd78e21-d34e-4212-ac4b-2ca86f2d8e13 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:23.164904 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4befa6fd-80ac-44f3-864a-944cd51d2003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:23.165611 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4befa6fd-80ac-44f3-864a-944cd51d2003 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:23.165897 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4befa6fd-80ac-44f3-864a-944cd51d2003 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:23.166743 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4befa6fd-80ac-44f3-864a-944cd51d2003 admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:23.167633 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4befa6fd-80ac-44f3-864a-944cd51d2003 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:23.174556 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4befa6fd-80ac-44f3-864a-944cd51d2003 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:23.216323 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 842/1676] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:53:23 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:53:23.217525 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2cb862-b8f6-4ad6-b58d-71efbbc830f8 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:23.241510 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2cb862-b8f6-4ad6-b58d-71efbbc830f8 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:23.258675 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ddaec0-c2cf-4e40-9683-59e706ee5c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:23.259719 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ddaec0-c2cf-4e40-9683-59e706ee5c76 admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:23.260069 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ddaec0-c2cf-4e40-9683-59e706ee5c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:23.260419 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ddaec0-c2cf-4e40-9683-59e706ee5c76 admin admin] PATH_INFO: `/v3/projects/1f18b138022642899dd02e83ed2025e8/users/cb524017c24e480292d3aef92f2efefe/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:23.297585 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:23.309565 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ddaec0-c2cf-4e40-9683-59e706ee5c76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f18b138022642899dd02e83ed2025e8, user_id=cb524017c24e480292d3aef92f2efefe, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:23.312751 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ddaec0-c2cf-4e40-9683-59e706ee5c76 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:23.461263 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 835/1677] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:53:23 2022] PUT /identity/v3/projects/1f18b138022642899dd02e83ed2025e8/users/cb524017c24e480292d3aef92f2efefe/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:23.474112 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7426fc7d-b59f-404a-936c-4666cb6f9f9b None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:23.474420 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7426fc7d-b59f-404a-936c-4666cb6f9f9b None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:23.479683 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7426fc7d-b59f-404a-936c-4666cb6f9f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:23.563840 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-7426fc7d-b59f-404a-936c-4666cb6f9f9b None None] MFA Rules not processed for user `cb524017c24e480292d3aef92f2efefe`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:23.565103 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-7426fc7d-b59f-404a-936c-4666cb6f9f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:23.755084 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 843/1678] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:23.760302 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ddaec0-c2cf-4e40-9683-59e706ee5c76 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:23.851497 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7426fc7d-b59f-404a-936c-4666cb6f9f9b None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:23.854163 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-7426fc7d-b59f-404a-936c-4666cb6f9f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:23.999689 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7426fc7d-b59f-404a-936c-4666cb6f9f9b None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:24.005623 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ddaec0-c2cf-4e40-9683-59e706ee5c76 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:24.006636 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1103c77e-254b-416c-8b52-82df72404118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:24.007758 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1103c77e-254b-416c-8b52-82df72404118 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:24.008151 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1103c77e-254b-416c-8b52-82df72404118 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:24.008533 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1103c77e-254b-416c-8b52-82df72404118 admin admin] PATH_INFO: `/v3/projects/185713cd153d4e9abe3afeabe9d9c723` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:24.010212 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1103c77e-254b-416c-8b52-82df72404118 admin admin] RBAC: Authorizing `identity:delete_project(project_id=185713cd153d4e9abe3afeabe9d9c723)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:24.024285 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c7d479-5722-4033-9b17-6ba66500aecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:24.027012 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7d479-5722-4033-9b17-6ba66500aecd admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:24.028500 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7d479-5722-4033-9b17-6ba66500aecd admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:24.028922 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7d479-5722-4033-9b17-6ba66500aecd admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:24.034650 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1103c77e-254b-416c-8b52-82df72404118 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:24.057766 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c7d479-5722-4033-9b17-6ba66500aecd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:24.066215 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c7d479-5722-4033-9b17-6ba66500aecd admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:24.244931 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 836/1679] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:53:23 2022] POST /identity/v3/projects => generated 387 bytes in 485 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:53:24.246838 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee86b5-969c-450a-944f-8a48b9308382 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:24.247331 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee86b5-969c-450a-944f-8a48b9308382 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:24.247716 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee86b5-969c-450a-944f-8a48b9308382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:24.291538 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-1103c77e-254b-416c-8b52-82df72404118 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '185713cd153d4e9abe3afeabe9d9c723', 'request_id': 'req-1103c77e-254b-416c-8b52-82df72404118'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:24.303782 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-08ee86b5-969c-450a-944f-8a48b9308382 None None] MFA Rules not processed for user `d10c68b7fd4446feb759832913fea3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:24.312507 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-08ee86b5-969c-450a-944f-8a48b9308382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:24.332567 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-1103c77e-254b-416c-8b52-82df72404118 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '185713cd153d4e9abe3afeabe9d9c723', 'request_id': 'req-1103c77e-254b-416c-8b52-82df72404118'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:24.378280 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:24.454647 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-1103c77e-254b-416c-8b52-82df72404118 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '185713cd153d4e9abe3afeabe9d9c723', 'request_id': 'req-1103c77e-254b-416c-8b52-82df72404118'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:24.476600 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-1103c77e-254b-416c-8b52-82df72404118 admin admin] The token cache is being invalidate because project 185713cd153d4e9abe3afeabe9d9c723 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:53:24.478218 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-1103c77e-254b-416c-8b52-82df72404118 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:24.491716 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-1103c77e-254b-416c-8b52-82df72404118 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:24.508400 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 844/1680] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:53:23 2022] DELETE /identity/v3/projects/185713cd153d4e9abe3afeabe9d9c723 => generated 0 bytes in 658 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:24.510289 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1103c77e-254b-416c-8b52-82df72404118 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:24.530719 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-1103c77e-254b-416c-8b52-82df72404118 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:24.646744 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 837/1681] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 401 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:24.648811 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed9c0c6-eef4-47e9-b626-a28af2a5654c None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:24.662840 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed9c0c6-eef4-47e9-b626-a28af2a5654c None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:24.663234 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed9c0c6-eef4-47e9-b626-a28af2a5654c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:24.723954 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-8ed9c0c6-eef4-47e9-b626-a28af2a5654c None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:24.728848 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-8ed9c0c6-eef4-47e9-b626-a28af2a5654c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:24.828678 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1103c77e-254b-416c-8b52-82df72404118 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:25.004174 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 838/1682] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:25.005497 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed9c0c6-eef4-47e9-b626-a28af2a5654c None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:25.006897 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-8ed9c0c6-eef4-47e9-b626-a28af2a5654c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:25.055737 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd2cc09-fd82-4e45-b01c-29c7887a64af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:25.056447 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2cc09-fd82-4e45-b01c-29c7887a64af admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:25.056732 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2cc09-fd82-4e45-b01c-29c7887a64af admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:25.057009 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2cc09-fd82-4e45-b01c-29c7887a64af admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:25.057714 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd2cc09-fd82-4e45-b01c-29c7887a64af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:25.064277 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd2cc09-fd82-4e45-b01c-29c7887a64af admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:25.170149 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 845/1683] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:53:24 2022] POST /identity/v3/users => generated 294 bytes in 661 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:53:25.174763 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20269e73-89f5-43bc-a1aa-a4fcf0061728 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:25.175082 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20269e73-89f5-43bc-a1aa-a4fcf0061728 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:25.175440 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20269e73-89f5-43bc-a1aa-a4fcf0061728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:25.216003 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed9c0c6-eef4-47e9-b626-a28af2a5654c None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:25.233815 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74881a4b-42c2-48c3-a3ff-eddb4e01ede4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bfdb655a2a44b33ab724daaf73a294b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:25.234500 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74881a4b-42c2-48c3-a3ff-eddb4e01ede4 service nova] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:25.235096 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74881a4b-42c2-48c3-a3ff-eddb4e01ede4 service nova] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:25.235283 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74881a4b-42c2-48c3-a3ff-eddb4e01ede4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:25.240206 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-20269e73-89f5-43bc-a1aa-a4fcf0061728 None None] MFA Rules not processed for user `4947466838974a71815561640fa55c14`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:25.242443 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-20269e73-89f5-43bc-a1aa-a4fcf0061728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:25.258536 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74881a4b-42c2-48c3-a3ff-eddb4e01ede4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:25.263116 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74881a4b-42c2-48c3-a3ff-eddb4e01ede4 service nova] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:25.280981 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 839/1684] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:53:25 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:53:25.282887 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a338ff-9ca7-4668-9e51-fefe05393aa6 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:25.283143 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a338ff-9ca7-4668-9e51-fefe05393aa6 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:25.283424 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a338ff-9ca7-4668-9e51-fefe05393aa6 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:25.288126 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 840/1685] 10.222.0.52 () {54 vars in 961 bytes} [Thu Jul 21 22:53:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:53:25.291695 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0babf414-892c-45ba-97fe-e0dc189aae62 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:25.292895 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0babf414-892c-45ba-97fe-e0dc189aae62 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:25.294296 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0babf414-892c-45ba-97fe-e0dc189aae62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:25.336364 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:25.364058 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-0babf414-892c-45ba-97fe-e0dc189aae62 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:25.365303 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-0babf414-892c-45ba-97fe-e0dc189aae62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:25.394532 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 841/1686] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:25.395554 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0babf414-892c-45ba-97fe-e0dc189aae62 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:25.414394 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 846/1687] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:25 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:25.416207 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369bfae6-b898-482c-81bc-aa1d87811ab3 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:25.416207 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369bfae6-b898-482c-81bc-aa1d87811ab3 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:25.416488 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369bfae6-b898-482c-81bc-aa1d87811ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:25.419617 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0babf414-892c-45ba-97fe-e0dc189aae62 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:25.445937 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c431a3-f4fd-44ab-990f-1f6cd84b91ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:25.446629 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c431a3-f4fd-44ab-990f-1f6cd84b91ab admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:25.446948 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c431a3-f4fd-44ab-990f-1f6cd84b91ab admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:25.447228 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c431a3-f4fd-44ab-990f-1f6cd84b91ab admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:25.447971 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c431a3-f4fd-44ab-990f-1f6cd84b91ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:25.449603 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c431a3-f4fd-44ab-990f-1f6cd84b91ab admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:25.484429 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 842/1688] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:53:25.489261 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c431a3-f4fd-44ab-990f-1f6cd84b91ab admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:25.523444 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c431a3-f4fd-44ab-990f-1f6cd84b91ab admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:25.530015 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-369bfae6-b898-482c-81bc-aa1d87811ab3 None None] MFA Rules not processed for user `10c0bed169344a77bfad29624c5e5a8e`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:25.539433 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-369bfae6-b898-482c-81bc-aa1d87811ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:25.542945 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babbbd4f-0f16-4142-a3fd-724b384ac551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:25.543861 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babbbd4f-0f16-4142-a3fd-724b384ac551 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:25.544635 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babbbd4f-0f16-4142-a3fd-724b384ac551 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:25.549369 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babbbd4f-0f16-4142-a3fd-724b384ac551 admin admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:25.550635 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babbbd4f-0f16-4142-a3fd-724b384ac551 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:53:25.551139 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babbbd4f-0f16-4142-a3fd-724b384ac551 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:25.559191 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babbbd4f-0f16-4142-a3fd-724b384ac551 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:25.600048 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 843/1689] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 22:53:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:53:25.607724 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727eb49e-4df4-44be-aedb-c5d2d7be10f6 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:25.610667 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727eb49e-4df4-44be-aedb-c5d2d7be10f6 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:25.611145 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727eb49e-4df4-44be-aedb-c5d2d7be10f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:25.669540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-727eb49e-4df4-44be-aedb-c5d2d7be10f6 None None] MFA Rules not processed for user `4947466838974a71815561640fa55c14`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:25.674128 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-727eb49e-4df4-44be-aedb-c5d2d7be10f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:25.706390 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 844/1690] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:25 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:25.707895 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727eb49e-4df4-44be-aedb-c5d2d7be10f6 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:25.710507 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 847/1691] 10.222.0.52 () {58 vars in 1058 bytes} [Thu Jul 21 22:53:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:25.713899 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369bfae6-b898-482c-81bc-aa1d87811ab3 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:25.748250 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369bfae6-b898-482c-81bc-aa1d87811ab3 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:25.749347 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727eb49e-4df4-44be-aedb-c5d2d7be10f6 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:25.757647 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4523a7-b3b3-4c44-9918-772109c3556e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:25.757647 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4523a7-b3b3-4c44-9918-772109c3556e admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:25.757647 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4523a7-b3b3-4c44-9918-772109c3556e admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:25.757647 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4523a7-b3b3-4c44-9918-772109c3556e admin admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:25.758126 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4523a7-b3b3-4c44-9918-772109c3556e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:25.763400 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef95f542-f835-41dd-8d30-42b76ded07a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:25.764331 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef95f542-f835-41dd-8d30-42b76ded07a6 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:25.765320 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4523a7-b3b3-4c44-9918-772109c3556e admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:25.766390 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef95f542-f835-41dd-8d30-42b76ded07a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:25.766676 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef95f542-f835-41dd-8d30-42b76ded07a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:25.767850 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef95f542-f835-41dd-8d30-42b76ded07a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:25.771310 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef95f542-f835-41dd-8d30-42b76ded07a6 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:25.777723 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:25.815513 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 845/1692] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:53:25.816633 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef95f542-f835-41dd-8d30-42b76ded07a6 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:25.818763 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-ef95f542-f835-41dd-8d30-42b76ded07a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:25.846684 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef95f542-f835-41dd-8d30-42b76ded07a6 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:25.854936 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8bbed4-2656-4a7a-a165-99a4f57885c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bfdb655a2a44b33ab724daaf73a294b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:25.855609 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8bbed4-2656-4a7a-a165-99a4f57885c3 service nova] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:25.855843 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8bbed4-2656-4a7a-a165-99a4f57885c3 service nova] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:25.856163 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8bbed4-2656-4a7a-a165-99a4f57885c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:25.867584 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8bbed4-2656-4a7a-a165-99a4f57885c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:25.870996 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8bbed4-2656-4a7a-a165-99a4f57885c3 service nova] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:25.895916 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 848/1693] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:53:25 2022] POST /identity/v3/projects => generated 381 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:53:25.906859 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 846/1694] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:53:25 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:53:25.909348 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8bbed4-2656-4a7a-a165-99a4f57885c3 service nova] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:25.928456 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4523a7-b3b3-4c44-9918-772109c3556e admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:25.945498 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8bbed4-2656-4a7a-a165-99a4f57885c3 service nova] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:26.098463 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd821323-5e6a-487f-b13a-087aa881f236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:26.099742 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd821323-5e6a-487f-b13a-087aa881f236 admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:26.100036 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd821323-5e6a-487f-b13a-087aa881f236 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:26.100317 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd821323-5e6a-487f-b13a-087aa881f236 admin admin] PATH_INFO: `/v3/projects/55615f9532474e8f9c4b9466e5781747/users/59dc37caf3194c40910ad294159d7859/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:26.145113 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd821323-5e6a-487f-b13a-087aa881f236 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55615f9532474e8f9c4b9466e5781747, user_id=59dc37caf3194c40910ad294159d7859, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:26.161879 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd821323-5e6a-487f-b13a-087aa881f236 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:26.190397 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4523a7-b3b3-4c44-9918-772109c3556e admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:26.206933 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3751d56-f184-462c-afaa-ae970cd8ca1a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10c0bed169344a77bfad29624c5e5a8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:26.213263 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3751d56-f184-462c-afaa-ae970cd8ca1a service placement] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:26.214048 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3751d56-f184-462c-afaa-ae970cd8ca1a service placement] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:26.214444 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3751d56-f184-462c-afaa-ae970cd8ca1a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:26.216718 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-f3751d56-f184-462c-afaa-ae970cd8ca1a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:26.232805 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3751d56-f184-462c-afaa-ae970cd8ca1a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:26.234405 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3751d56-f184-462c-afaa-ae970cd8ca1a service placement] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:26.267078 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 849/1695] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:53:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:53:26.267078 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3751d56-f184-462c-afaa-ae970cd8ca1a service placement] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:26.267518 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-f3751d56-f184-462c-afaa-ae970cd8ca1a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:26.286998 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3751d56-f184-462c-afaa-ae970cd8ca1a service placement] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:26.304124 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 847/1696] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:53:25 2022] PUT /identity/v3/projects/55615f9532474e8f9c4b9466e5781747/users/59dc37caf3194c40910ad294159d7859/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:26.306530 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd821323-5e6a-487f-b13a-087aa881f236 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:26.326191 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de25b833-dfb4-425c-a661-f78f67b104ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:26.326191 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de25b833-dfb4-425c-a661-f78f67b104ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:26.326191 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de25b833-dfb4-425c-a661-f78f67b104ce admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:26.326191 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de25b833-dfb4-425c-a661-f78f67b104ce admin admin] PATH_INFO: `/v3/users/b770572014374cfca1053bc07259a30e` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:26.327062 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de25b833-dfb4-425c-a661-f78f67b104ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=b770572014374cfca1053bc07259a30e)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:26.328257 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de25b833-dfb4-425c-a661-f78f67b104ce admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:26.536430 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd821323-5e6a-487f-b13a-087aa881f236 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:26.547529 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb7ab50-5348-478c-b3ad-417bfdc41b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:26.548574 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb7ab50-5348-478c-b3ad-417bfdc41b29 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:26.549041 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb7ab50-5348-478c-b3ad-417bfdc41b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:26.549879 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb7ab50-5348-478c-b3ad-417bfdc41b29 admin admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:26.550959 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb7ab50-5348-478c-b3ad-417bfdc41b29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:26.553420 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb7ab50-5348-478c-b3ad-417bfdc41b29 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:26.604947 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-de25b833-dfb4-425c-a661-f78f67b104ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b770572014374cfca1053bc07259a30e'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:26.617341 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-de25b833-dfb4-425c-a661-f78f67b104ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b770572014374cfca1053bc07259a30e'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:26.618436 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-de25b833-dfb4-425c-a661-f78f67b104ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b770572014374cfca1053bc07259a30e'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:26.643753 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-de25b833-dfb4-425c-a661-f78f67b104ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b770572014374cfca1053bc07259a30e'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:26.645048 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-de25b833-dfb4-425c-a661-f78f67b104ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b770572014374cfca1053bc07259a30e'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:26.684085 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 848/1697] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:53:26 2022] POST /identity/v3/users => generated 291 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:53:26.686328 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ed347-6a5a-4a55-8495-7d24f3c7a609 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:26.687858 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ed347-6a5a-4a55-8495-7d24f3c7a609 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:26.688420 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ed347-6a5a-4a55-8495-7d24f3c7a609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:26.740033 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 850/1698] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:53:26 2022] DELETE /identity/v3/users/b770572014374cfca1053bc07259a30e => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:26.741295 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de25b833-dfb4-425c-a661-f78f67b104ce admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:26.745913 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-de25b833-dfb4-425c-a661-f78f67b104ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:26.792015 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-431ed347-6a5a-4a55-8495-7d24f3c7a609 None None] MFA Rules not processed for user `59dc37caf3194c40910ad294159d7859`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:26.793095 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-431ed347-6a5a-4a55-8495-7d24f3c7a609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:26.957610 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de25b833-dfb4-425c-a661-f78f67b104ce admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:26.973351 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:26.984546 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac20a3e-2d65-4350-92c1-50f04f0e3696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:26.986455 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac20a3e-2d65-4350-92c1-50f04f0e3696 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:26.986752 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac20a3e-2d65-4350-92c1-50f04f0e3696 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:26.987157 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac20a3e-2d65-4350-92c1-50f04f0e3696 admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:26.991624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 849/1699] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:26.991802 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac20a3e-2d65-4350-92c1-50f04f0e3696 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:26.991882 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e46673-6c41-45c1-804e-a24ee55091a2 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:26.992799 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e46673-6c41-45c1-804e-a24ee55091a2 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:26.992905 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac20a3e-2d65-4350-92c1-50f04f0e3696 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:26.993115 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e46673-6c41-45c1-804e-a24ee55091a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:27.024752 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 851/1700] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:53:27.025432 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac20a3e-2d65-4350-92c1-50f04f0e3696 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:27.027257 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-3ac20a3e-2d65-4350-92c1-50f04f0e3696 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:27.039745 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-83e46673-6c41-45c1-804e-a24ee55091a2 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:27.040943 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-83e46673-6c41-45c1-804e-a24ee55091a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:27.071466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 850/1701] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:27.071466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e46673-6c41-45c1-804e-a24ee55091a2 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:27.083319 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac20a3e-2d65-4350-92c1-50f04f0e3696 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:27.092018 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e46673-6c41-45c1-804e-a24ee55091a2 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:27.100211 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d456f5-43f4-4434-80be-8454a269ed81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:27.102504 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d456f5-43f4-4434-80be-8454a269ed81 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:27.102504 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d456f5-43f4-4434-80be-8454a269ed81 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:27.102504 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d456f5-43f4-4434-80be-8454a269ed81 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:27.103300 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d456f5-43f4-4434-80be-8454a269ed81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:27.105940 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d456f5-43f4-4434-80be-8454a269ed81 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:27.108581 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227ce7a6-c961-4b69-a7d6-88f88f1d9d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:27.111674 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227ce7a6-c961-4b69-a7d6-88f88f1d9d5f admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:27.114366 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227ce7a6-c961-4b69-a7d6-88f88f1d9d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:27.115558 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227ce7a6-c961-4b69-a7d6-88f88f1d9d5f admin admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:27.117429 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227ce7a6-c961-4b69-a7d6-88f88f1d9d5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:27.119842 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227ce7a6-c961-4b69-a7d6-88f88f1d9d5f admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:27.152705 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 851/1702] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:53:27 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:53:27.153717 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d456f5-43f4-4434-80be-8454a269ed81 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:27.156048 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-d3d456f5-43f4-4434-80be-8454a269ed81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:27.255934 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d456f5-43f4-4434-80be-8454a269ed81 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:27.269718 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f6c7f-62ed-4910-bf2f-53a30182aed3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:27.271257 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f6c7f-62ed-4910-bf2f-53a30182aed3 service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:27.271257 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f6c7f-62ed-4910-bf2f-53a30182aed3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:27.271445 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f6c7f-62ed-4910-bf2f-53a30182aed3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:27.294331 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3f6c7f-62ed-4910-bf2f-53a30182aed3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:27.294516 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 852/1703] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:53:27 2022] POST /identity/v3/projects => generated 387 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:53:27.296022 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3f6c7f-62ed-4910-bf2f-53a30182aed3 service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:27.301673 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227ce7a6-c961-4b69-a7d6-88f88f1d9d5f admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:27.332832 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 852/1704] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:53:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:53:27.333894 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f6c7f-62ed-4910-bf2f-53a30182aed3 service neutron] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:27.369810 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f6c7f-62ed-4910-bf2f-53a30182aed3 service neutron] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:27.418330 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:27.441492 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227ce7a6-c961-4b69-a7d6-88f88f1d9d5f admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:27.449590 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc619f10-64a4-4a3a-a843-560f8c09dbec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:27.451481 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc619f10-64a4-4a3a-a843-560f8c09dbec admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:27.451767 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc619f10-64a4-4a3a-a843-560f8c09dbec admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:27.452053 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc619f10-64a4-4a3a-a843-560f8c09dbec admin admin] PATH_INFO: `/v3/projects/cfb8add25d6845599ea4122f1b0edce2/users/ba4ce5a421684893ab9046f698099600/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:27.466816 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:27.490146 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cf2427-1bad-4b15-ba30-31bf052e972f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:27.491497 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf2427-1bad-4b15-ba30-31bf052e972f admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:27.493275 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf2427-1bad-4b15-ba30-31bf052e972f admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:27.494803 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf2427-1bad-4b15-ba30-31bf052e972f admin admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:27.495551 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cf2427-1bad-4b15-ba30-31bf052e972f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:27.498560 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc619f10-64a4-4a3a-a843-560f8c09dbec admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfb8add25d6845599ea4122f1b0edce2, user_id=ba4ce5a421684893ab9046f698099600, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:27.503562 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cf2427-1bad-4b15-ba30-31bf052e972f admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:27.505903 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc619f10-64a4-4a3a-a843-560f8c09dbec admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:27.651833 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 853/1705] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:53:27 2022] PUT /identity/v3/projects/cfb8add25d6845599ea4122f1b0edce2/users/ba4ce5a421684893ab9046f698099600/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:27.653401 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910a53a-248b-462a-9bd6-fed495bec972 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:27.654068 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 853/1706] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:53:27 2022] POST /identity/v3/users => generated 300 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:53:27.655548 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cf2427-1bad-4b15-ba30-31bf052e972f admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:27.656053 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910a53a-248b-462a-9bd6-fed495bec972 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:27.656916 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910a53a-248b-462a-9bd6-fed495bec972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:27.699827 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cf2427-1bad-4b15-ba30-31bf052e972f admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:27.765142 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-4910a53a-248b-462a-9bd6-fed495bec972 None None] MFA Rules not processed for user `ef9ff560b7044ea2a18b3c3f8295937e`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:27.767151 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-4910a53a-248b-462a-9bd6-fed495bec972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:27.903573 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55112ec8-5550-4ec5-95fa-791021e32d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:27.904514 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55112ec8-5550-4ec5-95fa-791021e32d72 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:27.904964 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55112ec8-5550-4ec5-95fa-791021e32d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:27.905383 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55112ec8-5550-4ec5-95fa-791021e32d72 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:27.910886 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55112ec8-5550-4ec5-95fa-791021e32d72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:27.912747 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55112ec8-5550-4ec5-95fa-791021e32d72 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:27.939240 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 854/1707] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:53:27 2022] GET /identity/v3/roles => generated 1420 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:53:27.940963 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02798e59-2e3d-4878-905f-923e43f640c8 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:27.942656 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02798e59-2e3d-4878-905f-923e43f640c8 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:27.943171 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02798e59-2e3d-4878-905f-923e43f640c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:27.993476 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 854/1708] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:27.996192 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4910a53a-248b-462a-9bd6-fed495bec972 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:28.029834 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4910a53a-248b-462a-9bd6-fed495bec972 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:28.037736 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5489a6-7a81-4c34-aad4-a4cf5a429315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:28.039010 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5489a6-7a81-4c34-aad4-a4cf5a429315 admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:28.039340 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5489a6-7a81-4c34-aad4-a4cf5a429315 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:28.039624 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5489a6-7a81-4c34-aad4-a4cf5a429315 admin admin] PATH_INFO: `/v3/projects/2bdbcf25b6ee4341808ba52e490753b8/users/6ccc09a5063c41e0b367f987f83d4adf/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:28.042887 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-02798e59-2e3d-4878-905f-923e43f640c8 None None] MFA Rules not processed for user `ba4ce5a421684893ab9046f698099600`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:28.044296 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-02798e59-2e3d-4878-905f-923e43f640c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:28.074537 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5489a6-7a81-4c34-aad4-a4cf5a429315 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bdbcf25b6ee4341808ba52e490753b8, user_id=6ccc09a5063c41e0b367f987f83d4adf, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:28.083965 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5489a6-7a81-4c34-aad4-a4cf5a429315 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:28.200508 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 855/1709] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:53:27 2022] PUT /identity/v3/projects/2bdbcf25b6ee4341808ba52e490753b8/users/6ccc09a5063c41e0b367f987f83d4adf/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:28.205014 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5489a6-7a81-4c34-aad4-a4cf5a429315 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:28.208156 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-ab5489a6-7a81-4c34-aad4-a4cf5a429315 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:28.221990 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 855/1710] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:27 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:28.223798 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02798e59-2e3d-4878-905f-923e43f640c8 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:28.295085 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02798e59-2e3d-4878-905f-923e43f640c8 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:28.390947 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:28.461574 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5489a6-7a81-4c34-aad4-a4cf5a429315 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:28.473009 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca5475-e4a8-4796-8245-29b42d8e943f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:28.473726 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca5475-e4a8-4796-8245-29b42d8e943f admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:28.474013 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca5475-e4a8-4796-8245-29b42d8e943f admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:28.474295 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca5475-e4a8-4796-8245-29b42d8e943f admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:28.477977 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dca5475-e4a8-4796-8245-29b42d8e943f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:28.479904 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dca5475-e4a8-4796-8245-29b42d8e943f admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:28.489631 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d83bb51-7db3-4a6e-93f6-a355698aae3d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bfdb655a2a44b33ab724daaf73a294b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:28.490504 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d83bb51-7db3-4a6e-93f6-a355698aae3d service nova] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:28.492700 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d83bb51-7db3-4a6e-93f6-a355698aae3d service nova] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:28.493315 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d83bb51-7db3-4a6e-93f6-a355698aae3d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:28.523531 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 856/1711] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:53:28 2022] GET /identity/v3/roles => generated 1420 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:53:28.524876 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca5475-e4a8-4796-8245-29b42d8e943f admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:28.537662 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca5475-e4a8-4796-8245-29b42d8e943f admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:28.545922 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037b830a-7486-4c4b-9927-4d18a2c7720b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:28.547212 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b830a-7486-4c4b-9927-4d18a2c7720b admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:28.547683 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b830a-7486-4c4b-9927-4d18a2c7720b admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:28.548129 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b830a-7486-4c4b-9927-4d18a2c7720b admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:28.549236 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037b830a-7486-4c4b-9927-4d18a2c7720b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:28.552446 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037b830a-7486-4c4b-9927-4d18a2c7720b admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:28.667757 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d83bb51-7db3-4a6e-93f6-a355698aae3d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:28.669759 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d83bb51-7db3-4a6e-93f6-a355698aae3d service nova] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:28.690426 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 856/1712] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:53:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:53:28.692175 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feebcc2-da9b-4226-acdb-6853bfd0e33f None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:28.692646 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feebcc2-da9b-4226-acdb-6853bfd0e33f None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:28.692921 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feebcc2-da9b-4226-acdb-6853bfd0e33f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:28.721236 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-5feebcc2-da9b-4226-acdb-6853bfd0e33f None None] MFA Rules not processed for user `4947466838974a71815561640fa55c14`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:28.722506 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-5feebcc2-da9b-4226-acdb-6853bfd0e33f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:28.744736 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 857/1713] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:53:28 2022] POST /identity/v3/projects => generated 379 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:53:28.749373 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037b830a-7486-4c4b-9927-4d18a2c7720b admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:28.800219 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 857/1714] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:28 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:28.801330 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feebcc2-da9b-4226-acdb-6853bfd0e33f None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:28.849211 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feebcc2-da9b-4226-acdb-6853bfd0e33f None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:28.868278 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:29.035604 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87fa0ed-6809-41f8-997d-7201c8f1b18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:29.036276 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87fa0ed-6809-41f8-997d-7201c8f1b18c admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:29.036521 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87fa0ed-6809-41f8-997d-7201c8f1b18c admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:29.036864 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87fa0ed-6809-41f8-997d-7201c8f1b18c admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:29.037402 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87fa0ed-6809-41f8-997d-7201c8f1b18c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:29.050555 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87fa0ed-6809-41f8-997d-7201c8f1b18c admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:29.058201 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037b830a-7486-4c4b-9927-4d18a2c7720b admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:29.072112 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea6e720-48cd-4253-a3ac-765674393e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:29.073751 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea6e720-48cd-4253-a3ac-765674393e11 admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:29.074431 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea6e720-48cd-4253-a3ac-765674393e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:29.075092 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea6e720-48cd-4253-a3ac-765674393e11 admin admin] PATH_INFO: `/v3/projects/2bdbcf25b6ee4341808ba52e490753b8/users/6ccc09a5063c41e0b367f987f83d4adf/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:29.077499 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea6e720-48cd-4253-a3ac-765674393e11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bdbcf25b6ee4341808ba52e490753b8, user_id=6ccc09a5063c41e0b367f987f83d4adf, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:29.082752 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea6e720-48cd-4253-a3ac-765674393e11 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:29.120328 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 858/1715] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:53:28 2022] PUT /identity/v3/projects/2bdbcf25b6ee4341808ba52e490753b8/users/6ccc09a5063c41e0b367f987f83d4adf/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:29.120328 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea6e720-48cd-4253-a3ac-765674393e11 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:29.120328 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-cea6e720-48cd-4253-a3ac-765674393e11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:29.239660 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 858/1716] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:53:28 2022] POST /identity/v3/users => generated 296 bytes in 438 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:53:29.241828 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ea587-b9cd-465a-af5a-cf3155f173f3 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:29.242399 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ea587-b9cd-465a-af5a-cf3155f173f3 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:29.242692 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ea587-b9cd-465a-af5a-cf3155f173f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:29.290605 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-421ea587-b9cd-465a-af5a-cf3155f173f3 None None] MFA Rules not processed for user `6ccc09a5063c41e0b367f987f83d4adf`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:29.293150 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-421ea587-b9cd-465a-af5a-cf3155f173f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:29.354590 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea6e720-48cd-4253-a3ac-765674393e11 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:29.388267 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dac6ca-20a2-4b40-9913-f52dd96194c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:29.389440 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dac6ca-20a2-4b40-9913-f52dd96194c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:29.393778 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dac6ca-20a2-4b40-9913-f52dd96194c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:29.396189 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dac6ca-20a2-4b40-9913-f52dd96194c4 admin admin] PATH_INFO: `/v3/projects/7ee94dac76504026acfa65efdce67cd5` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:29.400826 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dac6ca-20a2-4b40-9913-f52dd96194c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ee94dac76504026acfa65efdce67cd5)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:29.407836 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dac6ca-20a2-4b40-9913-f52dd96194c4 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:29.470881 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 859/1717] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:29 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:29.471020 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421ea587-b9cd-465a-af5a-cf3155f173f3 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:29.489555 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421ea587-b9cd-465a-af5a-cf3155f173f3 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:29.507825 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7847e0e2-45ca-4b88-bbb4-4fcda916d6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:29.508523 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847e0e2-45ca-4b88-bbb4-4fcda916d6fc admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:29.508809 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847e0e2-45ca-4b88-bbb4-4fcda916d6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:29.509085 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847e0e2-45ca-4b88-bbb4-4fcda916d6fc admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:29.509964 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7847e0e2-45ca-4b88-bbb4-4fcda916d6fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:29.511654 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7847e0e2-45ca-4b88-bbb4-4fcda916d6fc admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:29.539221 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 860/1718] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:53:29 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:53:29.540345 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7847e0e2-45ca-4b88-bbb4-4fcda916d6fc admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:29.542352 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-7847e0e2-45ca-4b88-bbb4-4fcda916d6fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:29.609668 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7847e0e2-45ca-4b88-bbb4-4fcda916d6fc admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:29.629637 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-70dac6ca-20a2-4b40-9913-f52dd96194c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ee94dac76504026acfa65efdce67cd5', 'request_id': 'req-70dac6ca-20a2-4b40-9913-f52dd96194c4'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:29.643411 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-70dac6ca-20a2-4b40-9913-f52dd96194c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ee94dac76504026acfa65efdce67cd5', 'request_id': 'req-70dac6ca-20a2-4b40-9913-f52dd96194c4'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:29.687804 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d11135-283f-4f3f-bf77-3723ecb01a76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:29.688481 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d11135-283f-4f3f-bf77-3723ecb01a76 service cinder] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:29.688767 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d11135-283f-4f3f-bf77-3723ecb01a76 service cinder] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:29.689053 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d11135-283f-4f3f-bf77-3723ecb01a76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:29.763261 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-70dac6ca-20a2-4b40-9913-f52dd96194c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ee94dac76504026acfa65efdce67cd5', 'request_id': 'req-70dac6ca-20a2-4b40-9913-f52dd96194c4'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:29.779064 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-70dac6ca-20a2-4b40-9913-f52dd96194c4 admin admin] The token cache is being invalidate because project 7ee94dac76504026acfa65efdce67cd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:53:29.779963 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-70dac6ca-20a2-4b40-9913-f52dd96194c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:29.781539 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-70dac6ca-20a2-4b40-9913-f52dd96194c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:29.786573 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 859/1719] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:53:29 2022] DELETE /identity/v3/projects/7ee94dac76504026acfa65efdce67cd5 => generated 0 bytes in 676 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:29.790968 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dac6ca-20a2-4b40-9913-f52dd96194c4 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:29.793784 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-70dac6ca-20a2-4b40-9913-f52dd96194c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:29.800680 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d11135-283f-4f3f-bf77-3723ecb01a76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:29.816322 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d11135-283f-4f3f-bf77-3723ecb01a76 service cinder] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:29.849407 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 861/1720] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:53:29 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:53:29.855682 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a9242d-5751-4f1c-9283-a806330f153f None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:29.856685 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a9242d-5751-4f1c-9283-a806330f153f None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:29.857678 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a9242d-5751-4f1c-9283-a806330f153f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:29.905808 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-c2a9242d-5751-4f1c-9283-a806330f153f None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:29.908122 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-c2a9242d-5751-4f1c-9283-a806330f153f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:29.912829 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:30.011672 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dac6ca-20a2-4b40-9913-f52dd96194c4 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:30.033665 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ae3e9b-a852-4dd0-93a7-fb1d561648e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:30.038429 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae3e9b-a852-4dd0-93a7-fb1d561648e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:30.039639 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae3e9b-a852-4dd0-93a7-fb1d561648e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:30.041739 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae3e9b-a852-4dd0-93a7-fb1d561648e2 admin admin] PATH_INFO: `/v3/projects/87113ee29bca4ea7adb793bf89c4b392/users/7043162ba19349c98bbdfbb384a7cdf5/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:30.097682 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ae3e9b-a852-4dd0-93a7-fb1d561648e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87113ee29bca4ea7adb793bf89c4b392, user_id=7043162ba19349c98bbdfbb384a7cdf5, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:30.101556 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ae3e9b-a852-4dd0-93a7-fb1d561648e2 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:30.138239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 862/1721] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:30.140445 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2df64-34f4-4ca0-b808-cd41fc2b5d88 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:30.140445 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2df64-34f4-4ca0-b808-cd41fc2b5d88 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:30.140745 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2df64-34f4-4ca0-b808-cd41fc2b5d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:30.175824 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-50b2df64-34f4-4ca0-b808-cd41fc2b5d88 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:30.179425 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-50b2df64-34f4-4ca0-b808-cd41fc2b5d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:30.195132 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 860/1722] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:53:29 2022] PUT /identity/v3/projects/87113ee29bca4ea7adb793bf89c4b392/users/7043162ba19349c98bbdfbb384a7cdf5/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:30.199909 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ae3e9b-a852-4dd0-93a7-fb1d561648e2 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:30.224677 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 863/1723] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:53:30.226935 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b2df64-34f4-4ca0-b808-cd41fc2b5d88 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:30.251544 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b2df64-34f4-4ca0-b808-cd41fc2b5d88 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:30.364411 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ae3e9b-a852-4dd0-93a7-fb1d561648e2 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:30.375455 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d2839b-87ef-43e6-8732-4acb237b8912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:30.376152 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2839b-87ef-43e6-8732-4acb237b8912 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:30.376441 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2839b-87ef-43e6-8732-4acb237b8912 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:30.376721 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2839b-87ef-43e6-8732-4acb237b8912 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:30.377475 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d2839b-87ef-43e6-8732-4acb237b8912 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:30.381112 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d2839b-87ef-43e6-8732-4acb237b8912 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:30.396330 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9448dc98-f8f1-4994-a721-6f0fca679de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:30.397025 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9448dc98-f8f1-4994-a721-6f0fca679de9 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:30.397312 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9448dc98-f8f1-4994-a721-6f0fca679de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:30.398169 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9448dc98-f8f1-4994-a721-6f0fca679de9 admin admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:30.399219 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9448dc98-f8f1-4994-a721-6f0fca679de9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:53:30.399563 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9448dc98-f8f1-4994-a721-6f0fca679de9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:30.401201 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9448dc98-f8f1-4994-a721-6f0fca679de9 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:30.404978 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 861/1724] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:53:30 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:53:30.409204 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d2839b-87ef-43e6-8732-4acb237b8912 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:30.436638 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d2839b-87ef-43e6-8732-4acb237b8912 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:30.440922 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 864/1725] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 22:53:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:53:30.444634 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9448dc98-f8f1-4994-a721-6f0fca679de9 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:30.455927 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e5e348-96b0-4f46-841d-ffeee90178de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:30.456869 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5e348-96b0-4f46-841d-ffeee90178de admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:30.457157 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5e348-96b0-4f46-841d-ffeee90178de admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:30.457437 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5e348-96b0-4f46-841d-ffeee90178de admin admin] PATH_INFO: `/v3/projects/87113ee29bca4ea7adb793bf89c4b392/users/7043162ba19349c98bbdfbb384a7cdf5/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:30.461353 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e5e348-96b0-4f46-841d-ffeee90178de admin admin] RBAC: Authorizing `identity:create_grant(project_id=87113ee29bca4ea7adb793bf89c4b392, user_id=7043162ba19349c98bbdfbb384a7cdf5, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:30.464067 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e5e348-96b0-4f46-841d-ffeee90178de admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:30.471108 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9448dc98-f8f1-4994-a721-6f0fca679de9 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:30.508471 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:30.519702 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e1b4ae-90ad-4feb-8066-b537351c8f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:30.520402 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1b4ae-90ad-4feb-8066-b537351c8f71 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:30.521492 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1b4ae-90ad-4feb-8066-b537351c8f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:30.521785 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1b4ae-90ad-4feb-8066-b537351c8f71 admin admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:30.523753 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 862/1726] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:53:30 2022] PUT /identity/v3/projects/87113ee29bca4ea7adb793bf89c4b392/users/7043162ba19349c98bbdfbb384a7cdf5/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:30.532459 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e1b4ae-90ad-4feb-8066-b537351c8f71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:30.534155 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5596f2a-6a96-41c3-b615-98b0535b993a None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:30.535781 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e1b4ae-90ad-4feb-8066-b537351c8f71 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:30.536071 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5596f2a-6a96-41c3-b615-98b0535b993a None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:30.536635 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5596f2a-6a96-41c3-b615-98b0535b993a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:30.590982 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-f5596f2a-6a96-41c3-b615-98b0535b993a None None] MFA Rules not processed for user `7043162ba19349c98bbdfbb384a7cdf5`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:30.596223 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-f5596f2a-6a96-41c3-b615-98b0535b993a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:30.736048 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 865/1727] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:53:30 2022] POST /identity/v3/projects => generated 381 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:53:30.738198 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e1b4ae-90ad-4feb-8066-b537351c8f71 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:30.822071 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 863/1728] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:30 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:30.892494 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e1b4ae-90ad-4feb-8066-b537351c8f71 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:30.922652 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a912f0-6592-4e15-a00e-eb55d951bf50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:30.923409 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a912f0-6592-4e15-a00e-eb55d951bf50 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:30.923700 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a912f0-6592-4e15-a00e-eb55d951bf50 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:30.923984 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a912f0-6592-4e15-a00e-eb55d951bf50 admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:30.924702 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a912f0-6592-4e15-a00e-eb55d951bf50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:30.927005 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a912f0-6592-4e15-a00e-eb55d951bf50 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:31.115231 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 866/1729] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:53:30 2022] POST /identity/v3/users => generated 291 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:53:31.120068 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5596f2a-6a96-41c3-b615-98b0535b993a None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:31.301083 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5596f2a-6a96-41c3-b615-98b0535b993a None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:31.309746 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30665c9b-4294-4870-9abb-988699581aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:31.311116 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30665c9b-4294-4870-9abb-988699581aa0 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:31.311633 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30665c9b-4294-4870-9abb-988699581aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:31.313842 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30665c9b-4294-4870-9abb-988699581aa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:31.314995 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30665c9b-4294-4870-9abb-988699581aa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:31.317647 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30665c9b-4294-4870-9abb-988699581aa0 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:31.333829 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 864/1730] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:53:31 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:53:31.334616 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a912f0-6592-4e15-a00e-eb55d951bf50 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:31.362816 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a912f0-6592-4e15-a00e-eb55d951bf50 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:31.371406 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848b81a-fc3a-40ff-8d4c-1732fbf918bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:31.377982 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848b81a-fc3a-40ff-8d4c-1732fbf918bc admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:31.377982 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848b81a-fc3a-40ff-8d4c-1732fbf918bc admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:31.377982 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848b81a-fc3a-40ff-8d4c-1732fbf918bc admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:31.377982 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2848b81a-fc3a-40ff-8d4c-1732fbf918bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:31.377982 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2848b81a-fc3a-40ff-8d4c-1732fbf918bc admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:31.397139 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 867/1731] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:53:31 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:53:31.407701 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30665c9b-4294-4870-9abb-988699581aa0 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:31.489562 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30665c9b-4294-4870-9abb-988699581aa0 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:31.528272 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b2cdfe-b73d-4d15-8e31-8d0ed968f3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:31.532492 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b2cdfe-b73d-4d15-8e31-8d0ed968f3d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:31.534465 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b2cdfe-b73d-4d15-8e31-8d0ed968f3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:31.535060 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b2cdfe-b73d-4d15-8e31-8d0ed968f3d5 admin admin] PATH_INFO: `/v3/projects/347f9694cae2424d906cdf674c9fd72b/users/0f86bd7b724a4d2b8f577ff1a9c1001b/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:31.593881 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b2cdfe-b73d-4d15-8e31-8d0ed968f3d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=347f9694cae2424d906cdf674c9fd72b, user_id=0f86bd7b724a4d2b8f577ff1a9c1001b, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:31.603050 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b2cdfe-b73d-4d15-8e31-8d0ed968f3d5 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:31.721428 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 865/1732] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:53:31 2022] PUT /identity/v3/projects/347f9694cae2424d906cdf674c9fd72b/users/0f86bd7b724a4d2b8f577ff1a9c1001b/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:31.732871 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163b9a2-9833-4918-9b30-422d24d96576 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:31.733155 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163b9a2-9833-4918-9b30-422d24d96576 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:31.735042 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163b9a2-9833-4918-9b30-422d24d96576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:31.794836 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-2163b9a2-9833-4918-9b30-422d24d96576 None None] MFA Rules not processed for user `0f86bd7b724a4d2b8f577ff1a9c1001b`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:31.796386 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:31.796498 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-2163b9a2-9833-4918-9b30-422d24d96576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:31.945056 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 868/1733] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:31 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:53:31.949233 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b2cdfe-b73d-4d15-8e31-8d0ed968f3d5 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:32.158604 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b2cdfe-b73d-4d15-8e31-8d0ed968f3d5 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:32.186943 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec8a498-3988-4d1f-b438-762dd3bdc9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:32.187628 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec8a498-3988-4d1f-b438-762dd3bdc9b5 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:32.187914 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec8a498-3988-4d1f-b438-762dd3bdc9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:32.188188 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec8a498-3988-4d1f-b438-762dd3bdc9b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:32.189154 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec8a498-3988-4d1f-b438-762dd3bdc9b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:32.191199 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec8a498-3988-4d1f-b438-762dd3bdc9b5 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:32.288647 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2163b9a2-9833-4918-9b30-422d24d96576 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:32.290088 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-2163b9a2-9833-4918-9b30-422d24d96576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:32.417673 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2163b9a2-9833-4918-9b30-422d24d96576 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:32.454397 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 866/1734] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:53:31 2022] POST /identity/v3/projects => generated 383 bytes in 506 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:53:32.458547 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca48d5-dfee-419b-8c33-de531ef30bbe None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:32.461677 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca48d5-dfee-419b-8c33-de531ef30bbe None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:32.462125 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca48d5-dfee-419b-8c33-de531ef30bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:32.472518 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb68088-6568-473c-9e61-6421deba3218 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:32.474900 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb68088-6568-473c-9e61-6421deba3218 service cinder] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:32.478290 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb68088-6568-473c-9e61-6421deba3218 service cinder] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:32.478763 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb68088-6568-473c-9e61-6421deba3218 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:32.487153 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-abb68088-6568-473c-9e61-6421deba3218 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:32.557144 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-9dca48d5-dfee-419b-8c33-de531ef30bbe None None] MFA Rules not processed for user `d10c68b7fd4446feb759832913fea3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:32.559239 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-9dca48d5-dfee-419b-8c33-de531ef30bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:32.713400 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb68088-6568-473c-9e61-6421deba3218 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:32.715077 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb68088-6568-473c-9e61-6421deba3218 service cinder] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:32.734660 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 869/1735] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:53:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:53:32.734660 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb68088-6568-473c-9e61-6421deba3218 service cinder] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:32.749045 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb68088-6568-473c-9e61-6421deba3218 service cinder] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:32.761168 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 867/1736] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:32.764473 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180cc0e-73ce-42c8-a64b-0c2419215480 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:32.765981 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180cc0e-73ce-42c8-a64b-0c2419215480 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:32.766362 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180cc0e-73ce-42c8-a64b-0c2419215480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:32.943081 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-5180cc0e-73ce-42c8-a64b-0c2419215480 None None] MFA Rules not processed for user `63cc904578554aea84ff1917a5ed075e`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:32.945468 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-5180cc0e-73ce-42c8-a64b-0c2419215480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:32.972426 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acec18be-4628-4365-ae0f-81dea5e62fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:32.975124 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec18be-4628-4365-ae0f-81dea5e62fd8 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:32.979765 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec18be-4628-4365-ae0f-81dea5e62fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:32.980055 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec18be-4628-4365-ae0f-81dea5e62fd8 admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:32.980889 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acec18be-4628-4365-ae0f-81dea5e62fd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:33.007608 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acec18be-4628-4365-ae0f-81dea5e62fd8 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:33.198137 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 870/1737] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:53:32 2022] POST /identity/v3/users => generated 298 bytes in 466 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:53:33.202448 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acec18be-4628-4365-ae0f-81dea5e62fd8 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:33.226275 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:33.242286 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 868/1738] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jul 21 22:53:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 480 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:33.243700 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acec18be-4628-4365-ae0f-81dea5e62fd8 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:33.259874 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f31cc4-5861-4212-bb6c-70b0862270bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:33.260584 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f31cc4-5861-4212-bb6c-70b0862270bc admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:33.260867 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f31cc4-5861-4212-bb6c-70b0862270bc admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:33.261147 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f31cc4-5861-4212-bb6c-70b0862270bc admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:33.261904 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f31cc4-5861-4212-bb6c-70b0862270bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:33.263595 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f31cc4-5861-4212-bb6c-70b0862270bc admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:33.289769 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 871/1739] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:53:33 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:53:33.294523 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5180cc0e-73ce-42c8-a64b-0c2419215480 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:33.334577 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5180cc0e-73ce-42c8-a64b-0c2419215480 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:33.343391 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ebb029-fd4b-4f74-89f3-fa33356a6331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:33.344687 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ebb029-fd4b-4f74-89f3-fa33356a6331 admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:33.345052 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ebb029-fd4b-4f74-89f3-fa33356a6331 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:33.345574 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ebb029-fd4b-4f74-89f3-fa33356a6331 admin admin] PATH_INFO: `/v3/projects/0e9b97cb3a874fb69f018bf2fee6897a/users/0b5d955c1e844e4b8949d97419207227/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:33.404526 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ebb029-fd4b-4f74-89f3-fa33356a6331 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e9b97cb3a874fb69f018bf2fee6897a, user_id=0b5d955c1e844e4b8949d97419207227, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:33.418451 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ebb029-fd4b-4f74-89f3-fa33356a6331 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:33.597694 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f31cc4-5861-4212-bb6c-70b0862270bc admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:33.608671 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 869/1740] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:53:33 2022] PUT /identity/v3/projects/0e9b97cb3a874fb69f018bf2fee6897a/users/0b5d955c1e844e4b8949d97419207227/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:53:33.614208 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f31cc4-5861-4212-bb6c-70b0862270bc admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:33.679606 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ebb029-fd4b-4f74-89f3-fa33356a6331 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:33.682588 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-47ebb029-fd4b-4f74-89f3-fa33356a6331 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:33.783009 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a828b114-bd69-43e3-98bc-9679ecdbcf9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:33.783758 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828b114-bd69-43e3-98bc-9679ecdbcf9a admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:33.784054 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828b114-bd69-43e3-98bc-9679ecdbcf9a admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:33.784347 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828b114-bd69-43e3-98bc-9679ecdbcf9a admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:33.785273 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a828b114-bd69-43e3-98bc-9679ecdbcf9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:33.797942 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a828b114-bd69-43e3-98bc-9679ecdbcf9a admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:33.836826 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 872/1741] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:53:33 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:53:33.841552 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a828b114-bd69-43e3-98bc-9679ecdbcf9a admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:33.855522 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a828b114-bd69-43e3-98bc-9679ecdbcf9a admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:33.882409 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7962369-c28c-4e24-a5cc-e99a54f754ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:33.883671 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7962369-c28c-4e24-a5cc-e99a54f754ac admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:33.884130 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7962369-c28c-4e24-a5cc-e99a54f754ac admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:33.884702 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7962369-c28c-4e24-a5cc-e99a54f754ac admin admin] PATH_INFO: `/v3/projects/0e9b97cb3a874fb69f018bf2fee6897a/users/0b5d955c1e844e4b8949d97419207227/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:33.887919 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7962369-c28c-4e24-a5cc-e99a54f754ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e9b97cb3a874fb69f018bf2fee6897a, user_id=0b5d955c1e844e4b8949d97419207227, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:33.891030 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7962369-c28c-4e24-a5cc-e99a54f754ac admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:33.907697 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 873/1742] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:53:33 2022] PUT /identity/v3/projects/0e9b97cb3a874fb69f018bf2fee6897a/users/0b5d955c1e844e4b8949d97419207227/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:33.921761 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533801bf-1267-403c-bb37-6e979f6077a3 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:33.922051 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533801bf-1267-403c-bb37-6e979f6077a3 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:33.924566 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533801bf-1267-403c-bb37-6e979f6077a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:33.980392 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ebb029-fd4b-4f74-89f3-fa33356a6331 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:34.000265 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-533801bf-1267-403c-bb37-6e979f6077a3 None None] MFA Rules not processed for user `0b5d955c1e844e4b8949d97419207227`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:34.004328 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-533801bf-1267-403c-bb37-6e979f6077a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:34.071436 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e527a-d86e-44f5-8eb7-40940a046057 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba7eef4e1b27476ab4016449d9c3a4b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:34.072221 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e527a-d86e-44f5-8eb7-40940a046057 service swift] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:34.072221 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e527a-d86e-44f5-8eb7-40940a046057 service swift] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:34.072738 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e527a-d86e-44f5-8eb7-40940a046057 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:34.229288 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895e527a-d86e-44f5-8eb7-40940a046057 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:34.231127 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895e527a-d86e-44f5-8eb7-40940a046057 service swift] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:34.247552 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 874/1743] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:33 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:34.256397 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533801bf-1267-403c-bb37-6e979f6077a3 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:34.257968 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-533801bf-1267-403c-bb37-6e979f6077a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:34.266170 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 870/1744] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jul 21 22:53:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 21 22:53:34.439230 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533801bf-1267-403c-bb37-6e979f6077a3 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:34.474209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435cb04b-ae35-4bdd-90bb-364dc99ba525 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:34.476741 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435cb04b-ae35-4bdd-90bb-364dc99ba525 service cinder] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:34.477012 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435cb04b-ae35-4bdd-90bb-364dc99ba525 service cinder] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:34.482269 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435cb04b-ae35-4bdd-90bb-364dc99ba525 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:34.528469 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435cb04b-ae35-4bdd-90bb-364dc99ba525 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:34.540372 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435cb04b-ae35-4bdd-90bb-364dc99ba525 service cinder] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:34.589864 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 875/1745] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:53:34 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:53:34.843885 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e527a-d86e-44f5-8eb7-40940a046057 service swift] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:34.891512 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e527a-d86e-44f5-8eb7-40940a046057 service swift] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:34.906529 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82a614e-192b-476e-8bef-6394e874908e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:34.907796 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a614e-192b-476e-8bef-6394e874908e service cinder] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:34.908277 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a614e-192b-476e-8bef-6394e874908e service cinder] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:34.908892 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a614e-192b-476e-8bef-6394e874908e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:34.951230 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:35.110790 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82a614e-192b-476e-8bef-6394e874908e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:35.112845 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82a614e-192b-476e-8bef-6394e874908e service cinder] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:35.130623 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 871/1746] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:53:34 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:53:36.203661 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:36.340703 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f30a80-5934-4b48-87e6-27b4ffc705b1 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:36.341039 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f30a80-5934-4b48-87e6-27b4ffc705b1 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:36.341281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f30a80-5934-4b48-87e6-27b4ffc705b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:36.403119 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-e0f30a80-5934-4b48-87e6-27b4ffc705b1 None None] MFA Rules not processed for user `48c3e33f24ff4d66a72f06848d769ac7`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:36.406953 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-e0f30a80-5934-4b48-87e6-27b4ffc705b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:36.548824 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b341f8-56e6-4630-9caf-7319fe8f6ce7 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:36.550817 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b341f8-56e6-4630-9caf-7319fe8f6ce7 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:36.550817 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b341f8-56e6-4630-9caf-7319fe8f6ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:36.629636 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-88b341f8-56e6-4630-9caf-7319fe8f6ce7 None None] MFA Rules not processed for user `ef9ff560b7044ea2a18b3c3f8295937e`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:36.638261 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-88b341f8-56e6-4630-9caf-7319fe8f6ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:36.701147 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 876/1747] 10.222.0.33 () {58 vars in 1015 bytes} [Thu Jul 21 22:53:36 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:36.759632 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f30a80-5934-4b48-87e6-27b4ffc705b1 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:36.761715 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-e0f30a80-5934-4b48-87e6-27b4ffc705b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:36.795465 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f30a80-5934-4b48-87e6-27b4ffc705b1 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:36.821981 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3831d398-98b1-4821-8eb8-5e3496f74769 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:36.821981 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3831d398-98b1-4821-8eb8-5e3496f74769 service neutron] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:36.821981 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3831d398-98b1-4821-8eb8-5e3496f74769 service neutron] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:36.821981 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3831d398-98b1-4821-8eb8-5e3496f74769 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:36.838377 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 872/1748] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:53:36.846607 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3831d398-98b1-4821-8eb8-5e3496f74769 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:36.850689 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3831d398-98b1-4821-8eb8-5e3496f74769 service neutron] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:36.876741 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 877/1749] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:53:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:53:37.478881 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:38.277390 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:38.854605 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7f3bb3-b6d8-4711-97ee-86296921a4f2 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:38.855230 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7f3bb3-b6d8-4711-97ee-86296921a4f2 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:38.855339 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7f3bb3-b6d8-4711-97ee-86296921a4f2 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:38.856991 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 873/1750] 10.222.0.52 () {54 vars in 912 bytes} [Thu Jul 21 22:53:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:53:38.885139 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b34ee9a-4028-4c72-914a-54f33a0b01bf None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:38.885439 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b34ee9a-4028-4c72-914a-54f33a0b01bf None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:38.885716 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b34ee9a-4028-4c72-914a-54f33a0b01bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:38.934061 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-7b34ee9a-4028-4c72-914a-54f33a0b01bf None None] MFA Rules not processed for user `48c3e33f24ff4d66a72f06848d769ac7`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:38.936489 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-7b34ee9a-4028-4c72-914a-54f33a0b01bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:38.973331 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 878/1751] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jul 21 22:53:38 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:38.992247 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7f3bb3-b6d8-4711-97ee-86296921a4f2 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:39.038202 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7f3bb3-b6d8-4711-97ee-86296921a4f2 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:39.050201 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1ec818-f047-4075-8860-f595c2e70cb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:39.051776 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ec818-f047-4075-8860-f595c2e70cb1 service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:39.058972 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ec818-f047-4075-8860-f595c2e70cb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:39.060802 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ec818-f047-4075-8860-f595c2e70cb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:39.080664 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1ec818-f047-4075-8860-f595c2e70cb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:39.082608 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1ec818-f047-4075-8860-f595c2e70cb1 service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:39.161729 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 874/1752] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:53:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:53:39.409026 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:39.774169 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:39.904765 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b34ee9a-4028-4c72-914a-54f33a0b01bf None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:40.122873 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b34ee9a-4028-4c72-914a-54f33a0b01bf None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:40.154171 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc94dff8-e90f-43d1-8cd8-2651cb8737de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:40.169246 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc94dff8-e90f-43d1-8cd8-2651cb8737de admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:40.169650 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc94dff8-e90f-43d1-8cd8-2651cb8737de admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:40.171328 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc94dff8-e90f-43d1-8cd8-2651cb8737de admin admin] PATH_INFO: `/v3/users/0f86bd7b724a4d2b8f577ff1a9c1001b` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:40.188972 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc94dff8-e90f-43d1-8cd8-2651cb8737de admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f86bd7b724a4d2b8f577ff1a9c1001b)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:40.204802 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc94dff8-e90f-43d1-8cd8-2651cb8737de admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:40.279749 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0493e9be-640f-4a87-8096-ba4463470c46 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:40.288325 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0493e9be-640f-4a87-8096-ba4463470c46 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:40.288325 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0493e9be-640f-4a87-8096-ba4463470c46 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:40.288325 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 875/1753] 10.222.0.52 () {54 vars in 921 bytes} [Thu Jul 21 22:53:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:53:40.491192 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c5f6c7-fd6f-439b-b43f-3af5b07cf54a None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:40.491527 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c5f6c7-fd6f-439b-b43f-3af5b07cf54a None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:40.492037 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c5f6c7-fd6f-439b-b43f-3af5b07cf54a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:40.542718 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-cc94dff8-e90f-43d1-8cd8-2651cb8737de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f86bd7b724a4d2b8f577ff1a9c1001b'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:40.560988 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-cc94dff8-e90f-43d1-8cd8-2651cb8737de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f86bd7b724a4d2b8f577ff1a9c1001b'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:40.562102 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-cc94dff8-e90f-43d1-8cd8-2651cb8737de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f86bd7b724a4d2b8f577ff1a9c1001b'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:40.600367 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-e2c5f6c7-fd6f-439b-b43f-3af5b07cf54a None None] MFA Rules not processed for user `6bfdb655a2a44b33ab724daaf73a294b`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:40.609925 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-e2c5f6c7-fd6f-439b-b43f-3af5b07cf54a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:40.610137 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-cc94dff8-e90f-43d1-8cd8-2651cb8737de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f86bd7b724a4d2b8f577ff1a9c1001b'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:40.611182 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-cc94dff8-e90f-43d1-8cd8-2651cb8737de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f86bd7b724a4d2b8f577ff1a9c1001b'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:40.853889 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 879/1754] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:53:39 2022] DELETE /identity/v3/users/0f86bd7b724a4d2b8f577ff1a9c1001b => generated 0 bytes in 928 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:40.853889 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6052ef50-435f-45c5-9ba0-9fe95f091115 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:40.853889 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6052ef50-435f-45c5-9ba0-9fe95f091115 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:40.853889 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6052ef50-435f-45c5-9ba0-9fe95f091115 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:40.853889 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 880/1755] 10.222.0.52 () {54 vars in 921 bytes} [Thu Jul 21 22:53:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:53:40.861493 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b58a23-e1c9-4335-8e73-2753957723c5 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:40.862989 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b58a23-e1c9-4335-8e73-2753957723c5 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:40.865096 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b58a23-e1c9-4335-8e73-2753957723c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:40.927022 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 876/1756] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Jul 21 22:53:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 439 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:53:40.930511 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd4a4c0-ae5b-49e2-8910-225efc97fd70 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:40.930955 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd4a4c0-ae5b-49e2-8910-225efc97fd70 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:40.931100 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd4a4c0-ae5b-49e2-8910-225efc97fd70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:40.971086 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-86b58a23-e1c9-4335-8e73-2753957723c5 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:40.973887 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-86b58a23-e1c9-4335-8e73-2753957723c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:40.995870 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-bfd4a4c0-ae5b-49e2-8910-225efc97fd70 None None] MFA Rules not processed for user `6bfdb655a2a44b33ab724daaf73a294b`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:40.997299 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-bfd4a4c0-ae5b-49e2-8910-225efc97fd70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:41.143645 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 881/1757] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:41.195751 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 877/1758] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Jul 21 22:53:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:41.274859 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b58a23-e1c9-4335-8e73-2753957723c5 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:41.278741 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-86b58a23-e1c9-4335-8e73-2753957723c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:41.469871 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd4a4c0-ae5b-49e2-8910-225efc97fd70 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:41.471537 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-bfd4a4c0-ae5b-49e2-8910-225efc97fd70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:41.482422 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b58a23-e1c9-4335-8e73-2753957723c5 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:41.520682 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29188c9e-ab80-4f6a-9753-edc938742d4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:41.523555 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29188c9e-ab80-4f6a-9753-edc938742d4d service neutron] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:41.524822 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29188c9e-ab80-4f6a-9753-edc938742d4d service neutron] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:41.527237 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29188c9e-ab80-4f6a-9753-edc938742d4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:41.558913 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29188c9e-ab80-4f6a-9753-edc938742d4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:41.563986 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29188c9e-ab80-4f6a-9753-edc938742d4d service neutron] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:41.588966 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 882/1759] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:53:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:53:41.661127 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29188c9e-ab80-4f6a-9753-edc938742d4d service neutron] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:41.663958 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-29188c9e-ab80-4f6a-9753-edc938742d4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:41.697716 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd4a4c0-ae5b-49e2-8910-225efc97fd70 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:41.702336 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29188c9e-ab80-4f6a-9753-edc938742d4d service neutron] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:41.712847 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdf165b-f209-412a-87c3-ca61eaa1fb82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:41.721381 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdf165b-f209-412a-87c3-ca61eaa1fb82 service cinder] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:41.721381 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdf165b-f209-412a-87c3-ca61eaa1fb82 service cinder] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:41.721381 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdf165b-f209-412a-87c3-ca61eaa1fb82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:41.721381 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-8fdf165b-f209-412a-87c3-ca61eaa1fb82 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:41.729818 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea344c07-492a-4390-9750-6a90d6bcdddf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bfdb655a2a44b33ab724daaf73a294b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:41.730866 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea344c07-492a-4390-9750-6a90d6bcdddf service nova] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:41.731707 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea344c07-492a-4390-9750-6a90d6bcdddf service nova] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:41.732640 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea344c07-492a-4390-9750-6a90d6bcdddf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:41.770157 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea344c07-492a-4390-9750-6a90d6bcdddf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:41.775686 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea344c07-492a-4390-9750-6a90d6bcdddf service nova] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:41.812852 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 883/1760] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:53:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:53:41.859513 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdf165b-f209-412a-87c3-ca61eaa1fb82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:41.861571 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdf165b-f209-412a-87c3-ca61eaa1fb82 service cinder] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:41.885142 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 878/1761] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:53:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:53:41.890579 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:41.899129 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea344c07-492a-4390-9750-6a90d6bcdddf service nova] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:41.902665 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-ea344c07-492a-4390-9750-6a90d6bcdddf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:41.950425 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea344c07-492a-4390-9750-6a90d6bcdddf service nova] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:41.982639 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e55816-e335-4bb7-a7ea-dc77c0b1e5c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bfdb655a2a44b33ab724daaf73a294b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:41.985904 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e55816-e335-4bb7-a7ea-dc77c0b1e5c0 service nova] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:41.990520 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e55816-e335-4bb7-a7ea-dc77c0b1e5c0 service nova] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:41.991137 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e55816-e335-4bb7-a7ea-dc77c0b1e5c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:42.052699 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e55816-e335-4bb7-a7ea-dc77c0b1e5c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:42.065937 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e55816-e335-4bb7-a7ea-dc77c0b1e5c0 service nova] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:42.137149 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 884/1762] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:53:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:53:42.798633 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:43.721356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdf165b-f209-412a-87c3-ca61eaa1fb82 service cinder] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:43.721356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-8fdf165b-f209-412a-87c3-ca61eaa1fb82 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:43.735273 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdf165b-f209-412a-87c3-ca61eaa1fb82 service cinder] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:43.743026 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e971140-7988-47a8-9321-988791f9398c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:43.745362 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e971140-7988-47a8-9321-988791f9398c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:43.746193 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e971140-7988-47a8-9321-988791f9398c admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:43.747235 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e971140-7988-47a8-9321-988791f9398c admin admin] PATH_INFO: `/v3/projects/347f9694cae2424d906cdf674c9fd72b` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:43.749073 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e971140-7988-47a8-9321-988791f9398c admin admin] RBAC: Authorizing `identity:delete_project(project_id=347f9694cae2424d906cdf674c9fd72b)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:43.752282 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e971140-7988-47a8-9321-988791f9398c admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:43.796341 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:43.941455 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-2e971140-7988-47a8-9321-988791f9398c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '347f9694cae2424d906cdf674c9fd72b', 'request_id': 'req-2e971140-7988-47a8-9321-988791f9398c'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:43.952663 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-2e971140-7988-47a8-9321-988791f9398c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '347f9694cae2424d906cdf674c9fd72b', 'request_id': 'req-2e971140-7988-47a8-9321-988791f9398c'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:44.045424 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-2e971140-7988-47a8-9321-988791f9398c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '347f9694cae2424d906cdf674c9fd72b', 'request_id': 'req-2e971140-7988-47a8-9321-988791f9398c'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:44.067631 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-2e971140-7988-47a8-9321-988791f9398c admin admin] The token cache is being invalidate because project 347f9694cae2424d906cdf674c9fd72b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:53:44.071471 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-2e971140-7988-47a8-9321-988791f9398c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:44.074177 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-2e971140-7988-47a8-9321-988791f9398c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:44.084962 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 879/1763] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:53:43 2022] DELETE /identity/v3/projects/347f9694cae2424d906cdf674c9fd72b => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:44.085126 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e55816-e335-4bb7-a7ea-dc77c0b1e5c0 service nova] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:44.113329 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e55816-e335-4bb7-a7ea-dc77c0b1e5c0 service nova] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:44.302855 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3fea40-6c97-4e90-a73c-4510f7f86042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:44.306444 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3fea40-6c97-4e90-a73c-4510f7f86042 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:44.312227 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3fea40-6c97-4e90-a73c-4510f7f86042 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:44.312768 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3fea40-6c97-4e90-a73c-4510f7f86042 admin admin] PATH_INFO: `/v3/users/0b5d955c1e844e4b8949d97419207227` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:44.315516 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3fea40-6c97-4e90-a73c-4510f7f86042 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b5d955c1e844e4b8949d97419207227)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:44.318464 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3fea40-6c97-4e90-a73c-4510f7f86042 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:44.611250 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-6a3fea40-6c97-4e90-a73c-4510f7f86042 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b5d955c1e844e4b8949d97419207227'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:44.639458 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-6a3fea40-6c97-4e90-a73c-4510f7f86042 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b5d955c1e844e4b8949d97419207227'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:44.648580 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-6a3fea40-6c97-4e90-a73c-4510f7f86042 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b5d955c1e844e4b8949d97419207227'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:44.683244 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-6a3fea40-6c97-4e90-a73c-4510f7f86042 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b5d955c1e844e4b8949d97419207227'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:44.687796 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-6a3fea40-6c97-4e90-a73c-4510f7f86042 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b5d955c1e844e4b8949d97419207227'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:44.799404 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 885/1764] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:53:44 2022] DELETE /identity/v3/users/0b5d955c1e844e4b8949d97419207227 => generated 0 bytes in 716 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:44.975813 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:45.859818 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e971140-7988-47a8-9321-988791f9398c admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:45.861554 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-2e971140-7988-47a8-9321-988791f9398c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:45.879965 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e971140-7988-47a8-9321-988791f9398c admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:45.922598 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:46.027119 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee737f9-23ab-49d7-ae82-08928f85bde3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:46.030011 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee737f9-23ab-49d7-ae82-08928f85bde3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:46.030011 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee737f9-23ab-49d7-ae82-08928f85bde3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:46.030011 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee737f9-23ab-49d7-ae82-08928f85bde3 admin admin] PATH_INFO: `/v3/projects/0e9b97cb3a874fb69f018bf2fee6897a` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:46.033348 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee737f9-23ab-49d7-ae82-08928f85bde3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e9b97cb3a874fb69f018bf2fee6897a)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:46.035576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee737f9-23ab-49d7-ae82-08928f85bde3 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:46.127819 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:46.221104 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-8ee737f9-23ab-49d7-ae82-08928f85bde3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e9b97cb3a874fb69f018bf2fee6897a', 'request_id': 'req-8ee737f9-23ab-49d7-ae82-08928f85bde3'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:46.228479 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-8ee737f9-23ab-49d7-ae82-08928f85bde3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e9b97cb3a874fb69f018bf2fee6897a', 'request_id': 'req-8ee737f9-23ab-49d7-ae82-08928f85bde3'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:46.298979 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-8ee737f9-23ab-49d7-ae82-08928f85bde3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e9b97cb3a874fb69f018bf2fee6897a', 'request_id': 'req-8ee737f9-23ab-49d7-ae82-08928f85bde3'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:46.315339 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-8ee737f9-23ab-49d7-ae82-08928f85bde3 admin admin] The token cache is being invalidate because project 0e9b97cb3a874fb69f018bf2fee6897a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:53:46.315662 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-8ee737f9-23ab-49d7-ae82-08928f85bde3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:46.316535 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-8ee737f9-23ab-49d7-ae82-08928f85bde3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:46.321607 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 880/1765] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:53:45 2022] DELETE /identity/v3/projects/0e9b97cb3a874fb69f018bf2fee6897a => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:46.323652 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea2f03-84f7-47b8-aa07-c0e599e5fd59 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:46.324157 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea2f03-84f7-47b8-aa07-c0e599e5fd59 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:46.324649 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea2f03-84f7-47b8-aa07-c0e599e5fd59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:46.347346 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-64ea2f03-84f7-47b8-aa07-c0e599e5fd59 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:46.349277 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-64ea2f03-84f7-47b8-aa07-c0e599e5fd59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:46.435882 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 886/1766] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:46.437434 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543942d8-2d01-4e5f-b991-1935991cd314 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:46.437732 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543942d8-2d01-4e5f-b991-1935991cd314 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:46.438018 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543942d8-2d01-4e5f-b991-1935991cd314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:46.477998 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-543942d8-2d01-4e5f-b991-1935991cd314 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:46.479734 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-543942d8-2d01-4e5f-b991-1935991cd314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:46.488771 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 881/1767] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:53:46.493695 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ea2f03-84f7-47b8-aa07-c0e599e5fd59 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:46.504245 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ea2f03-84f7-47b8-aa07-c0e599e5fd59 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:46.512547 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e82296-24b0-4c27-a229-efe97e196105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:46.513244 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e82296-24b0-4c27-a229-efe97e196105 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:46.513528 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e82296-24b0-4c27-a229-efe97e196105 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:46.513808 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e82296-24b0-4c27-a229-efe97e196105 admin admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:46.514572 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e82296-24b0-4c27-a229-efe97e196105 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:53:46.514913 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e82296-24b0-4c27-a229-efe97e196105 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:46.516524 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e82296-24b0-4c27-a229-efe97e196105 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:46.535671 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 887/1768] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 22:53:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:53:46.536626 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543942d8-2d01-4e5f-b991-1935991cd314 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:46.547661 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543942d8-2d01-4e5f-b991-1935991cd314 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:46.553574 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6d3e80-f698-4648-aed1-e811bd26fe38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:46.554251 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6d3e80-f698-4648-aed1-e811bd26fe38 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:46.554531 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6d3e80-f698-4648-aed1-e811bd26fe38 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:46.554831 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6d3e80-f698-4648-aed1-e811bd26fe38 admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:46.555647 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6d3e80-f698-4648-aed1-e811bd26fe38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:46.557528 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6d3e80-f698-4648-aed1-e811bd26fe38 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:46.648696 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 882/1769] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:53:46 2022] POST /identity/v3/projects => generated 361 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:53:46.652606 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e82296-24b0-4c27-a229-efe97e196105 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:46.663908 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e82296-24b0-4c27-a229-efe97e196105 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:46.740019 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2477c18-b59b-4d4f-80d0-25391b762a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:46.742149 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2477c18-b59b-4d4f-80d0-25391b762a8c admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:46.742665 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2477c18-b59b-4d4f-80d0-25391b762a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:46.743229 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2477c18-b59b-4d4f-80d0-25391b762a8c admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:46.744962 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2477c18-b59b-4d4f-80d0-25391b762a8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:46.747300 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2477c18-b59b-4d4f-80d0-25391b762a8c admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:46.828072 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 888/1770] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:53:46 2022] POST /identity/v3/users => generated 281 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:53:46.832208 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6d3e80-f698-4648-aed1-e811bd26fe38 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:46.844017 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6d3e80-f698-4648-aed1-e811bd26fe38 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:46.854814 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe5c631-9c2b-4cfc-8e78-1f78e8200429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:46.855697 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5c631-9c2b-4cfc-8e78-1f78e8200429 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:46.856052 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5c631-9c2b-4cfc-8e78-1f78e8200429 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:46.856400 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5c631-9c2b-4cfc-8e78-1f78e8200429 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:46.857222 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe5c631-9c2b-4cfc-8e78-1f78e8200429 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:46.858939 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe5c631-9c2b-4cfc-8e78-1f78e8200429 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:46.870819 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 883/1771] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:53:46 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:53:46.874822 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2477c18-b59b-4d4f-80d0-25391b762a8c admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:46.885517 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2477c18-b59b-4d4f-80d0-25391b762a8c admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:46.891157 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5983ba-d5b7-491c-9cc2-fcecad018a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:46.891301 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5983ba-d5b7-491c-9cc2-fcecad018a3a admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:46.891605 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5983ba-d5b7-491c-9cc2-fcecad018a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:46.892089 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5983ba-d5b7-491c-9cc2-fcecad018a3a admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:46.892843 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5983ba-d5b7-491c-9cc2-fcecad018a3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:46.894444 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5983ba-d5b7-491c-9cc2-fcecad018a3a admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:46.905941 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 889/1772] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:53:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:53:46.910447 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe5c631-9c2b-4cfc-8e78-1f78e8200429 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:46.921532 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe5c631-9c2b-4cfc-8e78-1f78e8200429 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:46.926221 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf831862-6bca-49ed-a07d-6dbf2dbb5d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:46.927141 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf831862-6bca-49ed-a07d-6dbf2dbb5d79 admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:46.927423 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf831862-6bca-49ed-a07d-6dbf2dbb5d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:46.927731 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf831862-6bca-49ed-a07d-6dbf2dbb5d79 admin admin] PATH_INFO: `/v3/projects/b728805b1b3548149dd49f1024b7cf6a/users/3a1e6136136740dbb130329182121f8d/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:46.953783 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf831862-6bca-49ed-a07d-6dbf2dbb5d79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b728805b1b3548149dd49f1024b7cf6a, user_id=3a1e6136136740dbb130329182121f8d, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:46.956702 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf831862-6bca-49ed-a07d-6dbf2dbb5d79 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:47.050420 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 884/1773] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:53:46 2022] PUT /identity/v3/projects/b728805b1b3548149dd49f1024b7cf6a/users/3a1e6136136740dbb130329182121f8d/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:47.062083 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad9afa-46e1-4a53-a8c6-edb4e1434641 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:47.062376 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad9afa-46e1-4a53-a8c6-edb4e1434641 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:47.063148 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad9afa-46e1-4a53-a8c6-edb4e1434641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:47.095056 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-27ad9afa-46e1-4a53-a8c6-edb4e1434641 None None] MFA Rules not processed for user `3a1e6136136740dbb130329182121f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:47.096313 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-27ad9afa-46e1-4a53-a8c6-edb4e1434641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:47.178735 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 890/1774] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:47 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:47.233391 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf831862-6bca-49ed-a07d-6dbf2dbb5d79 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:47.235200 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-cf831862-6bca-49ed-a07d-6dbf2dbb5d79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:47.332825 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf831862-6bca-49ed-a07d-6dbf2dbb5d79 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:47.337870 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9728607-96c5-4705-ab2a-4437c411bd58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:47.338932 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9728607-96c5-4705-ab2a-4437c411bd58 service cinder] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:47.339374 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9728607-96c5-4705-ab2a-4437c411bd58 service cinder] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:47.339814 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9728607-96c5-4705-ab2a-4437c411bd58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:47.352289 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9728607-96c5-4705-ab2a-4437c411bd58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:47.354398 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9728607-96c5-4705-ab2a-4437c411bd58 service cinder] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:47.366513 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 885/1775] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:53:47 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:53:47.392133 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ad9afa-46e1-4a53-a8c6-edb4e1434641 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:47.421623 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9728607-96c5-4705-ab2a-4437c411bd58 service cinder] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:47.423068 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-a9728607-96c5-4705-ab2a-4437c411bd58 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:47.444131 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9728607-96c5-4705-ab2a-4437c411bd58 service cinder] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:47.453099 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321d48d-519d-4a59-8a4f-724627f00d5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:47.453807 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321d48d-519d-4a59-8a4f-724627f00d5d service cinder] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:47.455477 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321d48d-519d-4a59-8a4f-724627f00d5d service cinder] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:47.455886 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321d48d-519d-4a59-8a4f-724627f00d5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:47.461618 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-4321d48d-519d-4a59-8a4f-724627f00d5d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:47.492249 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:47.514553 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ad9afa-46e1-4a53-a8c6-edb4e1434641 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:47.519058 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a600c6-cee9-4212-a637-eec11ad68fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:47.520180 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a600c6-cee9-4212-a637-eec11ad68fa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:47.520474 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a600c6-cee9-4212-a637-eec11ad68fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:47.520601 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a600c6-cee9-4212-a637-eec11ad68fa8 admin admin] PATH_INFO: `/v3/users/3a1e6136136740dbb130329182121f8d` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:47.522023 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a600c6-cee9-4212-a637-eec11ad68fa8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a1e6136136740dbb130329182121f8d)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:47.525105 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a600c6-cee9-4212-a637-eec11ad68fa8 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:47.561938 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4321d48d-519d-4a59-8a4f-724627f00d5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:47.563481 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4321d48d-519d-4a59-8a4f-724627f00d5d service cinder] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:47.574402 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 886/1776] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:53:47 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:53:47.575486 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321d48d-519d-4a59-8a4f-724627f00d5d service cinder] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:47.576945 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-4321d48d-519d-4a59-8a4f-724627f00d5d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:47.595312 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321d48d-519d-4a59-8a4f-724627f00d5d service cinder] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:47.600414 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58afb510-684a-4c57-b535-5475d5046fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:47.601711 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58afb510-684a-4c57-b535-5475d5046fce admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:47.602035 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58afb510-684a-4c57-b535-5475d5046fce admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:47.602321 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58afb510-684a-4c57-b535-5475d5046fce admin admin] PATH_INFO: `/v3/users/cb524017c24e480292d3aef92f2efefe` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:47.603678 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58afb510-684a-4c57-b535-5475d5046fce admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb524017c24e480292d3aef92f2efefe)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:47.605821 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58afb510-684a-4c57-b535-5475d5046fce admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:47.637218 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:47.747943 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-c8a600c6-cee9-4212-a637-eec11ad68fa8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a1e6136136740dbb130329182121f8d'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:47.756544 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-c8a600c6-cee9-4212-a637-eec11ad68fa8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a1e6136136740dbb130329182121f8d'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:47.757228 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-c8a600c6-cee9-4212-a637-eec11ad68fa8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a1e6136136740dbb130329182121f8d'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:47.771610 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-c8a600c6-cee9-4212-a637-eec11ad68fa8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a1e6136136740dbb130329182121f8d'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:47.772601 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-c8a600c6-cee9-4212-a637-eec11ad68fa8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a1e6136136740dbb130329182121f8d'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:47.897920 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 891/1777] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:53:47 2022] DELETE /identity/v3/users/3a1e6136136740dbb130329182121f8d => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:47.899551 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db45e3f-edb7-44ee-a7a1-65b32b08e74a None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:47.900035 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db45e3f-edb7-44ee-a7a1-65b32b08e74a None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:47.900463 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db45e3f-edb7-44ee-a7a1-65b32b08e74a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:47.905371 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-58afb510-684a-4c57-b535-5475d5046fce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb524017c24e480292d3aef92f2efefe'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:47.916183 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-58afb510-684a-4c57-b535-5475d5046fce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb524017c24e480292d3aef92f2efefe'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:47.927184 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-58afb510-684a-4c57-b535-5475d5046fce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb524017c24e480292d3aef92f2efefe'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:47.927314 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-6db45e3f-edb7-44ee-a7a1-65b32b08e74a None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:47.927314 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-6db45e3f-edb7-44ee-a7a1-65b32b08e74a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:47.930387 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-58afb510-684a-4c57-b535-5475d5046fce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb524017c24e480292d3aef92f2efefe'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:47.984964 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 892/1778] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:47.995159 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db45e3f-edb7-44ee-a7a1-65b32b08e74a None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:47.996480 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-6db45e3f-edb7-44ee-a7a1-65b32b08e74a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:48.019072 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-58afb510-684a-4c57-b535-5475d5046fce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb524017c24e480292d3aef92f2efefe'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:48.026628 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 887/1779] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:53:47 2022] DELETE /identity/v3/users/cb524017c24e480292d3aef92f2efefe => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:48.031085 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40820ec7-e23d-4abf-8dd4-1bda6fdb9aab None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:48.031377 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40820ec7-e23d-4abf-8dd4-1bda6fdb9aab None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:48.031844 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40820ec7-e23d-4abf-8dd4-1bda6fdb9aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:48.050113 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-40820ec7-e23d-4abf-8dd4-1bda6fdb9aab None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:48.051476 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-40820ec7-e23d-4abf-8dd4-1bda6fdb9aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:48.058138 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db45e3f-edb7-44ee-a7a1-65b32b08e74a None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:48.111062 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657a11fd-3102-4565-b1e2-7f48dbcc512e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:48.111721 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 888/1780] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:53:48.111833 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a11fd-3102-4565-b1e2-7f48dbcc512e service neutron] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:48.112003 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a11fd-3102-4565-b1e2-7f48dbcc512e service neutron] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:48.112280 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a11fd-3102-4565-b1e2-7f48dbcc512e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:48.114221 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-657a11fd-3102-4565-b1e2-7f48dbcc512e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:48.121324 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40820ec7-e23d-4abf-8dd4-1bda6fdb9aab None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:48.122634 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-40820ec7-e23d-4abf-8dd4-1bda6fdb9aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:48.126586 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657a11fd-3102-4565-b1e2-7f48dbcc512e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:48.131254 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657a11fd-3102-4565-b1e2-7f48dbcc512e service neutron] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:48.137107 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40820ec7-e23d-4abf-8dd4-1bda6fdb9aab None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:48.140032 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 893/1781] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:53:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:53:48.141569 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1207d4-2552-46f2-9aa2-3affe4a91fa8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:48.142225 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1207d4-2552-46f2-9aa2-3affe4a91fa8 service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:48.142508 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1207d4-2552-46f2-9aa2-3affe4a91fa8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:48.142849 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1207d4-2552-46f2-9aa2-3affe4a91fa8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:48.152438 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1207d4-2552-46f2-9aa2-3affe4a91fa8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:48.153802 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1207d4-2552-46f2-9aa2-3affe4a91fa8 service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:48.163996 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 889/1782] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:53:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:53:48.604555 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657a11fd-3102-4565-b1e2-7f48dbcc512e service neutron] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:48.605379 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-657a11fd-3102-4565-b1e2-7f48dbcc512e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:48.676140 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657a11fd-3102-4565-b1e2-7f48dbcc512e service neutron] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:48.680841 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5450166-4028-43c3-af0e-056a46cc075e tempest-VolumeQuotaClassesTest-568216657 tempest-VolumeQuotaClassesTest-568216657-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7043162ba19349c98bbdfbb384a7cdf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87113ee29bca4ea7adb793bf89c4b392', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:48.681532 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5450166-4028-43c3-af0e-056a46cc075e tempest-VolumeQuotaClassesTest-568216657 tempest-VolumeQuotaClassesTest-568216657-project-admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:48.681800 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5450166-4028-43c3-af0e-056a46cc075e tempest-VolumeQuotaClassesTest-568216657 tempest-VolumeQuotaClassesTest-568216657-project-admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:48.682080 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5450166-4028-43c3-af0e-056a46cc075e tempest-VolumeQuotaClassesTest-568216657 tempest-VolumeQuotaClassesTest-568216657-project-admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:48.682887 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5450166-4028-43c3-af0e-056a46cc075e tempest-VolumeQuotaClassesTest-568216657 tempest-VolumeQuotaClassesTest-568216657-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:53:48.683199 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5450166-4028-43c3-af0e-056a46cc075e tempest-VolumeQuotaClassesTest-568216657 tempest-VolumeQuotaClassesTest-568216657-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:48.684803 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5450166-4028-43c3-af0e-056a46cc075e tempest-VolumeQuotaClassesTest-568216657 tempest-VolumeQuotaClassesTest-568216657-project-admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:48.697570 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 894/1783] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 22:53:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:53:48.701700 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1207d4-2552-46f2-9aa2-3affe4a91fa8 service neutron] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:48.713580 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1207d4-2552-46f2-9aa2-3affe4a91fa8 service neutron] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:48.719559 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a76459-f07e-400c-b960-71064f07feb2 tempest-VolumeQuotaClassesTest-568216657 tempest-VolumeQuotaClassesTest-568216657-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7043162ba19349c98bbdfbb384a7cdf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87113ee29bca4ea7adb793bf89c4b392', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:48.720493 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a76459-f07e-400c-b960-71064f07feb2 tempest-VolumeQuotaClassesTest-568216657 tempest-VolumeQuotaClassesTest-568216657-project-admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:48.720618 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a76459-f07e-400c-b960-71064f07feb2 tempest-VolumeQuotaClassesTest-568216657 tempest-VolumeQuotaClassesTest-568216657-project-admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:48.720895 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a76459-f07e-400c-b960-71064f07feb2 tempest-VolumeQuotaClassesTest-568216657 tempest-VolumeQuotaClassesTest-568216657-project-admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:48.721612 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a76459-f07e-400c-b960-71064f07feb2 tempest-VolumeQuotaClassesTest-568216657 tempest-VolumeQuotaClassesTest-568216657-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:48.723604 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a76459-f07e-400c-b960-71064f07feb2 tempest-VolumeQuotaClassesTest-568216657 tempest-VolumeQuotaClassesTest-568216657-project-admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:48.730061 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5450166-4028-43c3-af0e-056a46cc075e tempest-VolumeQuotaClassesTest-568216657 tempest-VolumeQuotaClassesTest-568216657-project-admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:48.734272 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-e5450166-4028-43c3-af0e-056a46cc075e tempest-VolumeQuotaClassesTest-568216657 tempest-VolumeQuotaClassesTest-568216657-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:48.750740 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5450166-4028-43c3-af0e-056a46cc075e tempest-VolumeQuotaClassesTest-568216657 tempest-VolumeQuotaClassesTest-568216657-project-admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:48.759550 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c790d09-0339-46cf-80b4-5fc7cf9cc086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:48.759550 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c790d09-0339-46cf-80b4-5fc7cf9cc086 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:48.759550 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c790d09-0339-46cf-80b4-5fc7cf9cc086 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:48.759550 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c790d09-0339-46cf-80b4-5fc7cf9cc086 admin admin] PATH_INFO: `/v3/projects/1f18b138022642899dd02e83ed2025e8` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:48.760201 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c790d09-0339-46cf-80b4-5fc7cf9cc086 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f18b138022642899dd02e83ed2025e8)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:48.760852 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c790d09-0339-46cf-80b4-5fc7cf9cc086 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:48.848152 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 890/1784] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:53:48 2022] POST /identity/v3/projects => generated 353 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:53:48.848152 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a76459-f07e-400c-b960-71064f07feb2 tempest-VolumeQuotaClassesTest-568216657 tempest-VolumeQuotaClassesTest-568216657-project-admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:48.849976 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-01a76459-f07e-400c-b960-71064f07feb2 tempest-VolumeQuotaClassesTest-568216657 tempest-VolumeQuotaClassesTest-568216657-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:48.896930 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-9c790d09-0339-46cf-80b4-5fc7cf9cc086 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f18b138022642899dd02e83ed2025e8', 'request_id': 'req-9c790d09-0339-46cf-80b4-5fc7cf9cc086'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:48.902822 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-9c790d09-0339-46cf-80b4-5fc7cf9cc086 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f18b138022642899dd02e83ed2025e8', 'request_id': 'req-9c790d09-0339-46cf-80b4-5fc7cf9cc086'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:48.949759 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a76459-f07e-400c-b960-71064f07feb2 tempest-VolumeQuotaClassesTest-568216657 tempest-VolumeQuotaClassesTest-568216657-project-admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:49.005325 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40434de6-ff3f-4657-8206-00b068f82cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:49.006532 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-9c790d09-0339-46cf-80b4-5fc7cf9cc086 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f18b138022642899dd02e83ed2025e8', 'request_id': 'req-9c790d09-0339-46cf-80b4-5fc7cf9cc086'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:49.006757 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40434de6-ff3f-4657-8206-00b068f82cb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:49.007074 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40434de6-ff3f-4657-8206-00b068f82cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:49.007366 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40434de6-ff3f-4657-8206-00b068f82cb2 admin admin] PATH_INFO: `/v3/projects/b728805b1b3548149dd49f1024b7cf6a` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:49.008555 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40434de6-ff3f-4657-8206-00b068f82cb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b728805b1b3548149dd49f1024b7cf6a)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:49.010539 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40434de6-ff3f-4657-8206-00b068f82cb2 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:49.019006 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-9c790d09-0339-46cf-80b4-5fc7cf9cc086 admin admin] The token cache is being invalidate because project 1f18b138022642899dd02e83ed2025e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:53:49.019288 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-9c790d09-0339-46cf-80b4-5fc7cf9cc086 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:49.019995 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-9c790d09-0339-46cf-80b4-5fc7cf9cc086 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:49.021834 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 895/1785] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:53:48 2022] DELETE /identity/v3/projects/1f18b138022642899dd02e83ed2025e8 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:49.022386 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c790d09-0339-46cf-80b4-5fc7cf9cc086 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:49.023699 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-9c790d09-0339-46cf-80b4-5fc7cf9cc086 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:49.087505 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c790d09-0339-46cf-80b4-5fc7cf9cc086 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:49.092108 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96258ad-90a9-48e7-a4ce-dda96cb84d99 tempest-VolumeQuotaClassesTest-568216657 tempest-VolumeQuotaClassesTest-568216657-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7043162ba19349c98bbdfbb384a7cdf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87113ee29bca4ea7adb793bf89c4b392', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:49.092766 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96258ad-90a9-48e7-a4ce-dda96cb84d99 tempest-VolumeQuotaClassesTest-568216657 tempest-VolumeQuotaClassesTest-568216657-project-admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:49.093300 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96258ad-90a9-48e7-a4ce-dda96cb84d99 tempest-VolumeQuotaClassesTest-568216657 tempest-VolumeQuotaClassesTest-568216657-project-admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:49.093584 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96258ad-90a9-48e7-a4ce-dda96cb84d99 tempest-VolumeQuotaClassesTest-568216657 tempest-VolumeQuotaClassesTest-568216657-project-admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:49.094358 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96258ad-90a9-48e7-a4ce-dda96cb84d99 tempest-VolumeQuotaClassesTest-568216657 tempest-VolumeQuotaClassesTest-568216657-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:53:49.094663 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96258ad-90a9-48e7-a4ce-dda96cb84d99 tempest-VolumeQuotaClassesTest-568216657 tempest-VolumeQuotaClassesTest-568216657-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:49.096281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96258ad-90a9-48e7-a4ce-dda96cb84d99 tempest-VolumeQuotaClassesTest-568216657 tempest-VolumeQuotaClassesTest-568216657-project-admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:49.107898 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 896/1786] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 22:53:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:53:49.108734 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96258ad-90a9-48e7-a4ce-dda96cb84d99 tempest-VolumeQuotaClassesTest-568216657 tempest-VolumeQuotaClassesTest-568216657-project-admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:49.110360 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-b96258ad-90a9-48e7-a4ce-dda96cb84d99 tempest-VolumeQuotaClassesTest-568216657 tempest-VolumeQuotaClassesTest-568216657-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:49.124127 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96258ad-90a9-48e7-a4ce-dda96cb84d99 tempest-VolumeQuotaClassesTest-568216657 tempest-VolumeQuotaClassesTest-568216657-project-admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:49.134942 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-40434de6-ff3f-4657-8206-00b068f82cb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b728805b1b3548149dd49f1024b7cf6a', 'request_id': 'req-40434de6-ff3f-4657-8206-00b068f82cb2'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:49.141550 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-40434de6-ff3f-4657-8206-00b068f82cb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b728805b1b3548149dd49f1024b7cf6a', 'request_id': 'req-40434de6-ff3f-4657-8206-00b068f82cb2'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:49.182411 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d10cad-d70b-4937-88d9-069c8e8ca565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:49.183288 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d10cad-d70b-4937-88d9-069c8e8ca565 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:49.183578 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d10cad-d70b-4937-88d9-069c8e8ca565 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:49.183859 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d10cad-d70b-4937-88d9-069c8e8ca565 admin admin] PATH_INFO: `/v3/users/59dc37caf3194c40910ad294159d7859` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:49.185059 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d10cad-d70b-4937-88d9-069c8e8ca565 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59dc37caf3194c40910ad294159d7859)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:49.187036 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d10cad-d70b-4937-88d9-069c8e8ca565 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:49.242185 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-40434de6-ff3f-4657-8206-00b068f82cb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b728805b1b3548149dd49f1024b7cf6a', 'request_id': 'req-40434de6-ff3f-4657-8206-00b068f82cb2'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:49.253865 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-40434de6-ff3f-4657-8206-00b068f82cb2 admin admin] The token cache is being invalidate because project b728805b1b3548149dd49f1024b7cf6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:53:49.254153 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-40434de6-ff3f-4657-8206-00b068f82cb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:49.254840 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-40434de6-ff3f-4657-8206-00b068f82cb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:49.257048 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 891/1787] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:53:48 2022] DELETE /identity/v3/projects/b728805b1b3548149dd49f1024b7cf6a => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:49.257877 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40434de6-ff3f-4657-8206-00b068f82cb2 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:49.259248 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-40434de6-ff3f-4657-8206-00b068f82cb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:49.265973 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:49.323858 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40434de6-ff3f-4657-8206-00b068f82cb2 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:49.328492 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5049e68e-444e-4367-99f0-8fa8fc7df2a2 tempest-VolumeQuotaClassesTest-568216657 tempest-VolumeQuotaClassesTest-568216657-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7043162ba19349c98bbdfbb384a7cdf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87113ee29bca4ea7adb793bf89c4b392', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:49.329348 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5049e68e-444e-4367-99f0-8fa8fc7df2a2 tempest-VolumeQuotaClassesTest-568216657 tempest-VolumeQuotaClassesTest-568216657-project-admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:49.329653 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5049e68e-444e-4367-99f0-8fa8fc7df2a2 tempest-VolumeQuotaClassesTest-568216657 tempest-VolumeQuotaClassesTest-568216657-project-admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:49.329956 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5049e68e-444e-4367-99f0-8fa8fc7df2a2 tempest-VolumeQuotaClassesTest-568216657 tempest-VolumeQuotaClassesTest-568216657-project-admin] PATH_INFO: `/v3/projects/8d5c142b8e0f423db1f71812c1f57807` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:49.331174 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5049e68e-444e-4367-99f0-8fa8fc7df2a2 tempest-VolumeQuotaClassesTest-568216657 tempest-VolumeQuotaClassesTest-568216657-project-admin] RBAC: Authorizing `identity:delete_project(project_id=8d5c142b8e0f423db1f71812c1f57807)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:49.333086 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5049e68e-444e-4367-99f0-8fa8fc7df2a2 tempest-VolumeQuotaClassesTest-568216657 tempest-VolumeQuotaClassesTest-568216657-project-admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:49.433616 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-53d10cad-d70b-4937-88d9-069c8e8ca565 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59dc37caf3194c40910ad294159d7859'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:49.440659 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-53d10cad-d70b-4937-88d9-069c8e8ca565 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59dc37caf3194c40910ad294159d7859'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:49.441312 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-53d10cad-d70b-4937-88d9-069c8e8ca565 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59dc37caf3194c40910ad294159d7859'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:49.447042 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-5049e68e-444e-4367-99f0-8fa8fc7df2a2 tempest-VolumeQuotaClassesTest-568216657 tempest-VolumeQuotaClassesTest-568216657-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d5c142b8e0f423db1f71812c1f57807', 'request_id': 'req-5049e68e-444e-4367-99f0-8fa8fc7df2a2'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:49.452851 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-5049e68e-444e-4367-99f0-8fa8fc7df2a2 tempest-VolumeQuotaClassesTest-568216657 tempest-VolumeQuotaClassesTest-568216657-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d5c142b8e0f423db1f71812c1f57807', 'request_id': 'req-5049e68e-444e-4367-99f0-8fa8fc7df2a2'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:49.454251 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-53d10cad-d70b-4937-88d9-069c8e8ca565 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59dc37caf3194c40910ad294159d7859'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:49.454704 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-53d10cad-d70b-4937-88d9-069c8e8ca565 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59dc37caf3194c40910ad294159d7859'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:49.498013 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:49.519921 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-5049e68e-444e-4367-99f0-8fa8fc7df2a2 tempest-VolumeQuotaClassesTest-568216657 tempest-VolumeQuotaClassesTest-568216657-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d5c142b8e0f423db1f71812c1f57807', 'request_id': 'req-5049e68e-444e-4367-99f0-8fa8fc7df2a2'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:49.534453 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-5049e68e-444e-4367-99f0-8fa8fc7df2a2 tempest-VolumeQuotaClassesTest-568216657 tempest-VolumeQuotaClassesTest-568216657-project-admin] The token cache is being invalidate because project 8d5c142b8e0f423db1f71812c1f57807 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:53:49.535470 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-5049e68e-444e-4367-99f0-8fa8fc7df2a2 tempest-VolumeQuotaClassesTest-568216657 tempest-VolumeQuotaClassesTest-568216657-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:49.535706 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-5049e68e-444e-4367-99f0-8fa8fc7df2a2 tempest-VolumeQuotaClassesTest-568216657 tempest-VolumeQuotaClassesTest-568216657-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:49.542744 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 892/1788] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:53:49 2022] DELETE /identity/v3/projects/8d5c142b8e0f423db1f71812c1f57807 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:49.545677 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9423ac4-0088-428f-a9ca-c027d46c0b8b None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:49.545677 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9423ac4-0088-428f-a9ca-c027d46c0b8b None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:49.545677 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9423ac4-0088-428f-a9ca-c027d46c0b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:49.584875 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 897/1789] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:53:49 2022] DELETE /identity/v3/users/59dc37caf3194c40910ad294159d7859 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:49.595257 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-f9423ac4-0088-428f-a9ca-c027d46c0b8b None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:49.598007 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-f9423ac4-0088-428f-a9ca-c027d46c0b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:49.688611 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 893/1790] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:49.696341 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a58d3ab-9325-402c-b4a8-14ddca1dd8bd None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:49.696853 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a58d3ab-9325-402c-b4a8-14ddca1dd8bd None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:49.696962 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a58d3ab-9325-402c-b4a8-14ddca1dd8bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:49.721020 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-3a58d3ab-9325-402c-b4a8-14ddca1dd8bd None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:49.722598 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-3a58d3ab-9325-402c-b4a8-14ddca1dd8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:49.735784 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 898/1791] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:49.740215 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9423ac4-0088-428f-a9ca-c027d46c0b8b None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:49.753206 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9423ac4-0088-428f-a9ca-c027d46c0b8b None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:49.765727 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f32f48-669f-47e2-9a9e-bcc0d7b2a39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:49.766548 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f32f48-669f-47e2-9a9e-bcc0d7b2a39c admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:49.766946 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f32f48-669f-47e2-9a9e-bcc0d7b2a39c admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:49.767751 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f32f48-669f-47e2-9a9e-bcc0d7b2a39c admin admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:49.768143 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f32f48-669f-47e2-9a9e-bcc0d7b2a39c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:53:49.768551 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f32f48-669f-47e2-9a9e-bcc0d7b2a39c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:49.770322 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f32f48-669f-47e2-9a9e-bcc0d7b2a39c admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:49.783957 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 894/1792] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 22:53:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:53:49.788863 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a58d3ab-9325-402c-b4a8-14ddca1dd8bd None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:49.798162 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a58d3ab-9325-402c-b4a8-14ddca1dd8bd None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:49.803109 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b6999f-8f00-40e4-bd6d-ee59e3bcb545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:49.804097 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b6999f-8f00-40e4-bd6d-ee59e3bcb545 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:49.804097 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b6999f-8f00-40e4-bd6d-ee59e3bcb545 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:49.811412 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b6999f-8f00-40e4-bd6d-ee59e3bcb545 admin admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:49.811412 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b6999f-8f00-40e4-bd6d-ee59e3bcb545 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:49.811412 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b6999f-8f00-40e4-bd6d-ee59e3bcb545 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:49.897724 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 899/1793] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:53:49 2022] POST /identity/v3/projects => generated 375 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:53:49.902311 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f32f48-669f-47e2-9a9e-bcc0d7b2a39c admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:49.914393 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f32f48-669f-47e2-9a9e-bcc0d7b2a39c admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:49.988177 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263c4fde-11ed-4780-acff-034c3395f876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:49.989065 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c4fde-11ed-4780-acff-034c3395f876 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:49.989356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c4fde-11ed-4780-acff-034c3395f876 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:49.989631 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c4fde-11ed-4780-acff-034c3395f876 admin admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:49.990919 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263c4fde-11ed-4780-acff-034c3395f876 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:49.993012 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263c4fde-11ed-4780-acff-034c3395f876 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:50.110972 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 895/1794] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:53:49 2022] POST /identity/v3/users => generated 288 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:53:50.118866 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b6999f-8f00-40e4-bd6d-ee59e3bcb545 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:50.125706 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b6999f-8f00-40e4-bd6d-ee59e3bcb545 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:50.130498 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f783b606-6d9b-4020-b7b2-54165d1e0599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:50.131256 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f783b606-6d9b-4020-b7b2-54165d1e0599 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:50.131541 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f783b606-6d9b-4020-b7b2-54165d1e0599 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:50.131819 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f783b606-6d9b-4020-b7b2-54165d1e0599 admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:50.132548 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f783b606-6d9b-4020-b7b2-54165d1e0599 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:50.134143 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f783b606-6d9b-4020-b7b2-54165d1e0599 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:50.145245 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 900/1795] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:53:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:53:50.150340 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263c4fde-11ed-4780-acff-034c3395f876 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:50.155624 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f783b606-6d9b-4020-b7b2-54165d1e0599 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:50.160441 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-f783b606-6d9b-4020-b7b2-54165d1e0599 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:50.165632 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263c4fde-11ed-4780-acff-034c3395f876 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:50.171155 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb081fa-eebd-47c5-9975-d9e97cadc3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:50.172892 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb081fa-eebd-47c5-9975-d9e97cadc3b3 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:50.173738 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb081fa-eebd-47c5-9975-d9e97cadc3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:50.174194 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb081fa-eebd-47c5-9975-d9e97cadc3b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:50.179639 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f783b606-6d9b-4020-b7b2-54165d1e0599 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:50.184227 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb081fa-eebd-47c5-9975-d9e97cadc3b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:50.184947 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb081fa-eebd-47c5-9975-d9e97cadc3b3 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:50.185784 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1185cb-3d2b-4e7c-9b89-d03e7e5aa245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:50.186766 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1185cb-3d2b-4e7c-9b89-d03e7e5aa245 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:50.187363 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1185cb-3d2b-4e7c-9b89-d03e7e5aa245 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:50.187715 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1185cb-3d2b-4e7c-9b89-d03e7e5aa245 admin admin] PATH_INFO: `/v3/projects/55615f9532474e8f9c4b9466e5781747` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:50.189157 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1185cb-3d2b-4e7c-9b89-d03e7e5aa245 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55615f9532474e8f9c4b9466e5781747)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:50.191234 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1185cb-3d2b-4e7c-9b89-d03e7e5aa245 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:50.202940 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 896/1796] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:53:50 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:53:50.202940 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb081fa-eebd-47c5-9975-d9e97cadc3b3 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:50.215077 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb081fa-eebd-47c5-9975-d9e97cadc3b3 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:50.221013 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab63338-68dd-4ee6-b8ac-024f44ffe5c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:50.222360 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab63338-68dd-4ee6-b8ac-024f44ffe5c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:50.222817 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab63338-68dd-4ee6-b8ac-024f44ffe5c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:50.225605 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab63338-68dd-4ee6-b8ac-024f44ffe5c7 admin admin] PATH_INFO: `/v3/projects/adf79b5233954a43a700b40b22b343b2/users/f8ce9364111548dcab0d4bb28fd3a37d/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:50.242299 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab63338-68dd-4ee6-b8ac-024f44ffe5c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=adf79b5233954a43a700b40b22b343b2, user_id=f8ce9364111548dcab0d4bb28fd3a37d, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:50.245234 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab63338-68dd-4ee6-b8ac-024f44ffe5c7 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:50.345124 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 897/1797] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:53:50 2022] PUT /identity/v3/projects/adf79b5233954a43a700b40b22b343b2/users/f8ce9364111548dcab0d4bb28fd3a37d/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:53:50.353981 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-ac1185cb-3d2b-4e7c-9b89-d03e7e5aa245 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55615f9532474e8f9c4b9466e5781747', 'request_id': 'req-ac1185cb-3d2b-4e7c-9b89-d03e7e5aa245'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:50.357323 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d0a960-3df7-4e53-805a-f774cbef587d None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:50.357618 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d0a960-3df7-4e53-805a-f774cbef587d None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:50.357895 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d0a960-3df7-4e53-805a-f774cbef587d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:50.359671 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-ac1185cb-3d2b-4e7c-9b89-d03e7e5aa245 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55615f9532474e8f9c4b9466e5781747', 'request_id': 'req-ac1185cb-3d2b-4e7c-9b89-d03e7e5aa245'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:50.388026 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-78d0a960-3df7-4e53-805a-f774cbef587d None None] MFA Rules not processed for user `f8ce9364111548dcab0d4bb28fd3a37d`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:50.389488 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-78d0a960-3df7-4e53-805a-f774cbef587d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:50.441244 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-ac1185cb-3d2b-4e7c-9b89-d03e7e5aa245 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55615f9532474e8f9c4b9466e5781747', 'request_id': 'req-ac1185cb-3d2b-4e7c-9b89-d03e7e5aa245'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:50.453768 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-ac1185cb-3d2b-4e7c-9b89-d03e7e5aa245 admin admin] The token cache is being invalidate because project 55615f9532474e8f9c4b9466e5781747 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:53:50.457492 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 898/1798] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:53:50.457592 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-ac1185cb-3d2b-4e7c-9b89-d03e7e5aa245 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:50.457677 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d0a960-3df7-4e53-805a-f774cbef587d None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:50.457926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-ac1185cb-3d2b-4e7c-9b89-d03e7e5aa245 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:50.458601 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-78d0a960-3df7-4e53-805a-f774cbef587d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:50.460098 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 901/1799] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:53:50 2022] DELETE /identity/v3/projects/55615f9532474e8f9c4b9466e5781747 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:50.463770 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1185cb-3d2b-4e7c-9b89-d03e7e5aa245 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:50.465164 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-ac1185cb-3d2b-4e7c-9b89-d03e7e5aa245 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:50.526263 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d0a960-3df7-4e53-805a-f774cbef587d None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:50.530019 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1185cb-3d2b-4e7c-9b89-d03e7e5aa245 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:50.531122 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df737b8-6475-4410-b66e-4c10ee0e949d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:50.531999 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df737b8-6475-4410-b66e-4c10ee0e949d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:50.532282 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df737b8-6475-4410-b66e-4c10ee0e949d admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:50.532559 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df737b8-6475-4410-b66e-4c10ee0e949d admin admin] PATH_INFO: `/v3/users/ba4ce5a421684893ab9046f698099600` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:50.533756 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df737b8-6475-4410-b66e-4c10ee0e949d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba4ce5a421684893ab9046f698099600)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:50.534966 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f509cd-34f4-47aa-9478-6b38e37b168a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:50.535699 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df737b8-6475-4410-b66e-4c10ee0e949d admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:50.535838 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f509cd-34f4-47aa-9478-6b38e37b168a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:50.536121 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f509cd-34f4-47aa-9478-6b38e37b168a admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:50.536398 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f509cd-34f4-47aa-9478-6b38e37b168a admin admin] PATH_INFO: `/v3/users/6ccc09a5063c41e0b367f987f83d4adf` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:50.537605 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f509cd-34f4-47aa-9478-6b38e37b168a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ccc09a5063c41e0b367f987f83d4adf)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:50.539602 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f509cd-34f4-47aa-9478-6b38e37b168a admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:50.725525 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-3df737b8-6475-4410-b66e-4c10ee0e949d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba4ce5a421684893ab9046f698099600'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:50.732729 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-3df737b8-6475-4410-b66e-4c10ee0e949d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba4ce5a421684893ab9046f698099600'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:50.733472 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-3df737b8-6475-4410-b66e-4c10ee0e949d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba4ce5a421684893ab9046f698099600'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:50.748961 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-3df737b8-6475-4410-b66e-4c10ee0e949d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba4ce5a421684893ab9046f698099600'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:50.773572 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-c0f509cd-34f4-47aa-9478-6b38e37b168a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ccc09a5063c41e0b367f987f83d4adf'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:50.782083 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-c0f509cd-34f4-47aa-9478-6b38e37b168a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ccc09a5063c41e0b367f987f83d4adf'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:50.783013 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-c0f509cd-34f4-47aa-9478-6b38e37b168a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ccc09a5063c41e0b367f987f83d4adf'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:50.796371 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-c0f509cd-34f4-47aa-9478-6b38e37b168a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ccc09a5063c41e0b367f987f83d4adf'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:50.797066 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-c0f509cd-34f4-47aa-9478-6b38e37b168a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ccc09a5063c41e0b367f987f83d4adf'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:50.830363 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-3df737b8-6475-4410-b66e-4c10ee0e949d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba4ce5a421684893ab9046f698099600'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:50.836974 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 899/1800] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:53:50 2022] DELETE /identity/v3/users/ba4ce5a421684893ab9046f698099600 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:50.840947 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df737b8-6475-4410-b66e-4c10ee0e949d admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:50.842293 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-3df737b8-6475-4410-b66e-4c10ee0e949d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:50.898222 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df737b8-6475-4410-b66e-4c10ee0e949d admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:50.901876 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a6aabb-76fd-484f-bb82-4031c929e8d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:50.902521 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a6aabb-76fd-484f-bb82-4031c929e8d7 service cinder] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:50.902825 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a6aabb-76fd-484f-bb82-4031c929e8d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:50.903104 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a6aabb-76fd-484f-bb82-4031c929e8d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:50.904864 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-a6a6aabb-76fd-484f-bb82-4031c929e8d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:50.916658 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 902/1801] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:53:50 2022] DELETE /identity/v3/users/6ccc09a5063c41e0b367f987f83d4adf => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:53:50.921034 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a398b62-aaa0-427f-8520-3ebf49b72f02 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:50.921387 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a398b62-aaa0-427f-8520-3ebf49b72f02 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:50.921807 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a398b62-aaa0-427f-8520-3ebf49b72f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:50.940049 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-3a398b62-aaa0-427f-8520-3ebf49b72f02 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:50.941349 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-3a398b62-aaa0-427f-8520-3ebf49b72f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:50.982912 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a6aabb-76fd-484f-bb82-4031c929e8d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:50.982912 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a6aabb-76fd-484f-bb82-4031c929e8d7 service cinder] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:50.992331 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 900/1802] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:53:50 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:53:51.013867 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 903/1803] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:51.023957 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a6aabb-76fd-484f-bb82-4031c929e8d7 service cinder] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:51.025509 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-a6a6aabb-76fd-484f-bb82-4031c929e8d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:51.096151 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a6aabb-76fd-484f-bb82-4031c929e8d7 service cinder] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:51.150508 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7514a9a-5965-4e15-957a-31d31805fbb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:51.151842 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7514a9a-5965-4e15-957a-31d31805fbb6 service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:51.152257 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7514a9a-5965-4e15-957a-31d31805fbb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:51.152603 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7514a9a-5965-4e15-957a-31d31805fbb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:51.163719 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7514a9a-5965-4e15-957a-31d31805fbb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:51.166188 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7514a9a-5965-4e15-957a-31d31805fbb6 service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:51.179761 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 901/1804] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:53:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:53:51.270369 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:51.537216 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a398b62-aaa0-427f-8520-3ebf49b72f02 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:51.539323 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-3a398b62-aaa0-427f-8520-3ebf49b72f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:51.552016 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a398b62-aaa0-427f-8520-3ebf49b72f02 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:51.557713 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af7e234-92eb-4b5d-9716-ef573d50e8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:51.559158 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af7e234-92eb-4b5d-9716-ef573d50e8d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:51.559158 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af7e234-92eb-4b5d-9716-ef573d50e8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:51.559357 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af7e234-92eb-4b5d-9716-ef573d50e8d9 admin admin] PATH_INFO: `/v3/projects/2bdbcf25b6ee4341808ba52e490753b8` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:51.560553 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af7e234-92eb-4b5d-9716-ef573d50e8d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bdbcf25b6ee4341808ba52e490753b8)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:51.562669 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af7e234-92eb-4b5d-9716-ef573d50e8d9 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:51.691205 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-9af7e234-92eb-4b5d-9716-ef573d50e8d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bdbcf25b6ee4341808ba52e490753b8', 'request_id': 'req-9af7e234-92eb-4b5d-9716-ef573d50e8d9'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:51.698183 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-9af7e234-92eb-4b5d-9716-ef573d50e8d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bdbcf25b6ee4341808ba52e490753b8', 'request_id': 'req-9af7e234-92eb-4b5d-9716-ef573d50e8d9'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:51.784502 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-9af7e234-92eb-4b5d-9716-ef573d50e8d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bdbcf25b6ee4341808ba52e490753b8', 'request_id': 'req-9af7e234-92eb-4b5d-9716-ef573d50e8d9'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:51.797712 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-9af7e234-92eb-4b5d-9716-ef573d50e8d9 admin admin] The token cache is being invalidate because project 2bdbcf25b6ee4341808ba52e490753b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:53:51.798078 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-9af7e234-92eb-4b5d-9716-ef573d50e8d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:51.798760 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-9af7e234-92eb-4b5d-9716-ef573d50e8d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:51.800633 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 904/1805] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:53:51 2022] DELETE /identity/v3/projects/2bdbcf25b6ee4341808ba52e490753b8 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:51.807126 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e8074-28ce-417f-ab77-329b1221d3a8 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:51.811845 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e8074-28ce-417f-ab77-329b1221d3a8 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:51.812125 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:51.812242 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e8074-28ce-417f-ab77-329b1221d3a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:51.818502 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af7e234-92eb-4b5d-9716-ef573d50e8d9 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:51.822759 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-9af7e234-92eb-4b5d-9716-ef573d50e8d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:51.840513 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-482e8074-28ce-417f-ab77-329b1221d3a8 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:51.841861 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-482e8074-28ce-417f-ab77-329b1221d3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:51.906069 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af7e234-92eb-4b5d-9716-ef573d50e8d9 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:51.911071 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1301105-d23b-4e34-9e07-f5256a8b52f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:51.911972 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1301105-d23b-4e34-9e07-f5256a8b52f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:51.912422 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1301105-d23b-4e34-9e07-f5256a8b52f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:51.912696 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1301105-d23b-4e34-9e07-f5256a8b52f7 admin admin] PATH_INFO: `/v3/projects/cfb8add25d6845599ea4122f1b0edce2` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:51.914118 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1301105-d23b-4e34-9e07-f5256a8b52f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfb8add25d6845599ea4122f1b0edce2)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:51.916132 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1301105-d23b-4e34-9e07-f5256a8b52f7 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:51.923909 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 902/1806] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:51.933329 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac3112-4008-4c7a-b076-3331f8cd7687 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:51.933807 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac3112-4008-4c7a-b076-3331f8cd7687 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:51.934099 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac3112-4008-4c7a-b076-3331f8cd7687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:51.958941 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-2bac3112-4008-4c7a-b076-3331f8cd7687 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:51.960228 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-2bac3112-4008-4c7a-b076-3331f8cd7687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:51.968777 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 903/1807] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:51.973029 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bac3112-4008-4c7a-b076-3331f8cd7687 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:51.983155 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bac3112-4008-4c7a-b076-3331f8cd7687 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:51.988025 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c036a770-8e92-40f2-a749-ce794b956434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:51.988683 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036a770-8e92-40f2-a749-ce794b956434 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:51.988969 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036a770-8e92-40f2-a749-ce794b956434 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:51.989245 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036a770-8e92-40f2-a749-ce794b956434 admin admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:51.990495 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c036a770-8e92-40f2-a749-ce794b956434 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:53:51.990834 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c036a770-8e92-40f2-a749-ce794b956434 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:51.992435 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c036a770-8e92-40f2-a749-ce794b956434 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:52.004237 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 904/1808] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 22:53:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:53:52.009005 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c036a770-8e92-40f2-a749-ce794b956434 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:52.019345 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c036a770-8e92-40f2-a749-ce794b956434 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:52.026583 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed33dd78-dadb-4670-8a0b-ca11e2aef165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:52.027477 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed33dd78-dadb-4670-8a0b-ca11e2aef165 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:52.027851 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed33dd78-dadb-4670-8a0b-ca11e2aef165 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:52.028114 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed33dd78-dadb-4670-8a0b-ca11e2aef165 admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:52.029014 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed33dd78-dadb-4670-8a0b-ca11e2aef165 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:52.030993 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed33dd78-dadb-4670-8a0b-ca11e2aef165 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:52.048344 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-e1301105-d23b-4e34-9e07-f5256a8b52f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfb8add25d6845599ea4122f1b0edce2', 'request_id': 'req-e1301105-d23b-4e34-9e07-f5256a8b52f7'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:52.054347 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-e1301105-d23b-4e34-9e07-f5256a8b52f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfb8add25d6845599ea4122f1b0edce2', 'request_id': 'req-e1301105-d23b-4e34-9e07-f5256a8b52f7'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:52.127274 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-e1301105-d23b-4e34-9e07-f5256a8b52f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfb8add25d6845599ea4122f1b0edce2', 'request_id': 'req-e1301105-d23b-4e34-9e07-f5256a8b52f7'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:52.134984 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 905/1809] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:53:52 2022] POST /identity/v3/projects => generated 393 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:53:52.139105 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed33dd78-dadb-4670-8a0b-ca11e2aef165 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:52.140304 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-e1301105-d23b-4e34-9e07-f5256a8b52f7 admin admin] The token cache is being invalidate because project cfb8add25d6845599ea4122f1b0edce2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:53:52.140591 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-e1301105-d23b-4e34-9e07-f5256a8b52f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:52.141200 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-e1301105-d23b-4e34-9e07-f5256a8b52f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:52.142810 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 905/1810] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:53:51 2022] DELETE /identity/v3/projects/cfb8add25d6845599ea4122f1b0edce2 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:52.146505 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1301105-d23b-4e34-9e07-f5256a8b52f7 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:52.148086 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-e1301105-d23b-4e34-9e07-f5256a8b52f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:52.202894 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed33dd78-dadb-4670-8a0b-ca11e2aef165 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:52.207657 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a600a-83bd-49bf-929b-50e3bc7c9d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:52.208332 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a600a-83bd-49bf-929b-50e3bc7c9d6f admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:52.208651 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a600a-83bd-49bf-929b-50e3bc7c9d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:52.209012 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a600a-83bd-49bf-929b-50e3bc7c9d6f admin admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:52.209620 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514a600a-83bd-49bf-929b-50e3bc7c9d6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:52.211565 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514a600a-83bd-49bf-929b-50e3bc7c9d6f admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:52.214011 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1301105-d23b-4e34-9e07-f5256a8b52f7 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:52.219961 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c27e7ea-f48d-4440-a730-d82f3bc2910e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:52.220827 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c27e7ea-f48d-4440-a730-d82f3bc2910e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:52.221106 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c27e7ea-f48d-4440-a730-d82f3bc2910e admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:52.221410 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c27e7ea-f48d-4440-a730-d82f3bc2910e admin admin] PATH_INFO: `/v3/users/7043162ba19349c98bbdfbb384a7cdf5` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:52.222650 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c27e7ea-f48d-4440-a730-d82f3bc2910e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7043162ba19349c98bbdfbb384a7cdf5)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:52.224919 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c27e7ea-f48d-4440-a730-d82f3bc2910e admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:52.292584 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 906/1811] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:53:52 2022] POST /identity/v3/users => generated 297 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:53:52.296649 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a600a-83bd-49bf-929b-50e3bc7c9d6f admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:52.298414 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-514a600a-83bd-49bf-929b-50e3bc7c9d6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:52.310986 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:52.311725 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a600a-83bd-49bf-929b-50e3bc7c9d6f admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:52.316588 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff63c9c7-068b-4b35-9564-357ed1edcbeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:52.317263 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff63c9c7-068b-4b35-9564-357ed1edcbeb admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:52.317637 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff63c9c7-068b-4b35-9564-357ed1edcbeb admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:52.317854 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff63c9c7-068b-4b35-9564-357ed1edcbeb admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:52.319583 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff63c9c7-068b-4b35-9564-357ed1edcbeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:52.321184 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff63c9c7-068b-4b35-9564-357ed1edcbeb admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:52.333708 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 907/1812] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:53:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:53:52.335785 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff63c9c7-068b-4b35-9564-357ed1edcbeb admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:52.345641 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff63c9c7-068b-4b35-9564-357ed1edcbeb admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:52.350557 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc075c-6f1f-4728-b74d-c77d798ebb5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:52.351252 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc075c-6f1f-4728-b74d-c77d798ebb5b admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:52.351542 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc075c-6f1f-4728-b74d-c77d798ebb5b admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:52.351836 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc075c-6f1f-4728-b74d-c77d798ebb5b admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:52.352547 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fc075c-6f1f-4728-b74d-c77d798ebb5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:52.354134 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fc075c-6f1f-4728-b74d-c77d798ebb5b admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:52.364798 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 908/1813] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:53:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:53:52.368759 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc075c-6f1f-4728-b74d-c77d798ebb5b admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:52.378704 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc075c-6f1f-4728-b74d-c77d798ebb5b admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:52.385921 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8544c2-0832-4a75-9fbd-1147c2773a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:52.386859 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8544c2-0832-4a75-9fbd-1147c2773a70 admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:52.387153 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8544c2-0832-4a75-9fbd-1147c2773a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:52.387423 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8544c2-0832-4a75-9fbd-1147c2773a70 admin admin] PATH_INFO: `/v3/projects/726799b752574673895da15b9b6b4aa7/users/0c1f5011fd3444b082b54fc1bd60dff3/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:52.403020 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8544c2-0832-4a75-9fbd-1147c2773a70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=726799b752574673895da15b9b6b4aa7, user_id=0c1f5011fd3444b082b54fc1bd60dff3, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:52.405710 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8544c2-0832-4a75-9fbd-1147c2773a70 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:52.432071 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-9c27e7ea-f48d-4440-a730-d82f3bc2910e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7043162ba19349c98bbdfbb384a7cdf5'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:52.439039 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-9c27e7ea-f48d-4440-a730-d82f3bc2910e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7043162ba19349c98bbdfbb384a7cdf5'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:52.439695 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-9c27e7ea-f48d-4440-a730-d82f3bc2910e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7043162ba19349c98bbdfbb384a7cdf5'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:52.452833 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-9c27e7ea-f48d-4440-a730-d82f3bc2910e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7043162ba19349c98bbdfbb384a7cdf5'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:52.453565 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-9c27e7ea-f48d-4440-a730-d82f3bc2910e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7043162ba19349c98bbdfbb384a7cdf5'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:52.499544 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 909/1814] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:53:52 2022] PUT /identity/v3/projects/726799b752574673895da15b9b6b4aa7/users/0c1f5011fd3444b082b54fc1bd60dff3/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:53:52.505798 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a8a6c8-9c12-4fe7-b128-c7ceff10a945 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:52.506094 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a8a6c8-9c12-4fe7-b128-c7ceff10a945 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:52.506399 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a8a6c8-9c12-4fe7-b128-c7ceff10a945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:52.516644 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 906/1815] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:53:52 2022] DELETE /identity/v3/users/7043162ba19349c98bbdfbb384a7cdf5 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:53:52.541201 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-69a8a6c8-9c12-4fe7-b128-c7ceff10a945 None None] MFA Rules not processed for user `0c1f5011fd3444b082b54fc1bd60dff3`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:52.542489 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-69a8a6c8-9c12-4fe7-b128-c7ceff10a945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:52.621410 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 910/1816] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:53:52.625852 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c27e7ea-f48d-4440-a730-d82f3bc2910e admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:52.627273 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-9c27e7ea-f48d-4440-a730-d82f3bc2910e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:52.700299 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c27e7ea-f48d-4440-a730-d82f3bc2910e admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:52.705130 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0676c736-67ca-46c0-8288-5f6b69d133c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:52.705800 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0676c736-67ca-46c0-8288-5f6b69d133c5 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:52.706086 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0676c736-67ca-46c0-8288-5f6b69d133c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:52.706368 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0676c736-67ca-46c0-8288-5f6b69d133c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:52.707132 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0676c736-67ca-46c0-8288-5f6b69d133c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:52.709086 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0676c736-67ca-46c0-8288-5f6b69d133c5 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:52.802865 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 907/1817] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:53:52 2022] POST /identity/v3/projects => generated 393 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:53:52.806472 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a8a6c8-9c12-4fe7-b128-c7ceff10a945 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:52.817590 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a8a6c8-9c12-4fe7-b128-c7ceff10a945 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:52.877447 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c40294-4c52-43c2-a62f-a90615364e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:52.878129 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c40294-4c52-43c2-a62f-a90615364e70 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:52.878409 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c40294-4c52-43c2-a62f-a90615364e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:52.878686 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c40294-4c52-43c2-a62f-a90615364e70 admin admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:52.879409 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c40294-4c52-43c2-a62f-a90615364e70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:52.881393 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c40294-4c52-43c2-a62f-a90615364e70 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:52.944007 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0676c736-67ca-46c0-8288-5f6b69d133c5 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:52.946283 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-0676c736-67ca-46c0-8288-5f6b69d133c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:52.960931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0676c736-67ca-46c0-8288-5f6b69d133c5 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:52.967061 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3018d5fe-2d82-4ca9-bc5d-537a3aa61114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:52.968277 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3018d5fe-2d82-4ca9-bc5d-537a3aa61114 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:52.968674 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3018d5fe-2d82-4ca9-bc5d-537a3aa61114 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:52.968922 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3018d5fe-2d82-4ca9-bc5d-537a3aa61114 admin admin] PATH_INFO: `/v3/projects/87113ee29bca4ea7adb793bf89c4b392` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:52.970296 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3018d5fe-2d82-4ca9-bc5d-537a3aa61114 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87113ee29bca4ea7adb793bf89c4b392)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:52.977271 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3018d5fe-2d82-4ca9-bc5d-537a3aa61114 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:53.001685 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 911/1818] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:53:52 2022] POST /identity/v3/users => generated 303 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:53:53.008037 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c40294-4c52-43c2-a62f-a90615364e70 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:53.017788 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c40294-4c52-43c2-a62f-a90615364e70 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:53.023030 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1a554f-bdde-4296-a4ce-d51595e2357d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:53.023690 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a554f-bdde-4296-a4ce-d51595e2357d admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:53.028261 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a554f-bdde-4296-a4ce-d51595e2357d admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:53.028261 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a554f-bdde-4296-a4ce-d51595e2357d admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:53.028261 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1a554f-bdde-4296-a4ce-d51595e2357d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:53.028261 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1a554f-bdde-4296-a4ce-d51595e2357d admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:53.037307 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 912/1819] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:53:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:53:53.041197 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1a554f-bdde-4296-a4ce-d51595e2357d admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:53.051193 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1a554f-bdde-4296-a4ce-d51595e2357d admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:53.056351 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339643b5-9dc6-4847-a508-94268c32afc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:53.057303 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339643b5-9dc6-4847-a508-94268c32afc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:53.057584 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339643b5-9dc6-4847-a508-94268c32afc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:53.057857 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339643b5-9dc6-4847-a508-94268c32afc4 admin admin] PATH_INFO: `/v3/projects/8cf245b95d8b4061a01cb6dfb6dd767c/users/7849a9af90544991a0957ba4c714002f/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:53.074259 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339643b5-9dc6-4847-a508-94268c32afc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cf245b95d8b4061a01cb6dfb6dd767c, user_id=7849a9af90544991a0957ba4c714002f, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:53.077903 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339643b5-9dc6-4847-a508-94268c32afc4 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:53.179424 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 913/1820] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:53:53 2022] PUT /identity/v3/projects/8cf245b95d8b4061a01cb6dfb6dd767c/users/7849a9af90544991a0957ba4c714002f/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:53.183430 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339643b5-9dc6-4847-a508-94268c32afc4 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:53.192237 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-3018d5fe-2d82-4ca9-bc5d-537a3aa61114 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87113ee29bca4ea7adb793bf89c4b392', 'request_id': 'req-3018d5fe-2d82-4ca9-bc5d-537a3aa61114'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:53.198469 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-3018d5fe-2d82-4ca9-bc5d-537a3aa61114 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87113ee29bca4ea7adb793bf89c4b392', 'request_id': 'req-3018d5fe-2d82-4ca9-bc5d-537a3aa61114'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:53.237083 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:53.250062 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339643b5-9dc6-4847-a508-94268c32afc4 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:53.270096 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-3018d5fe-2d82-4ca9-bc5d-537a3aa61114 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87113ee29bca4ea7adb793bf89c4b392', 'request_id': 'req-3018d5fe-2d82-4ca9-bc5d-537a3aa61114'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:53.282978 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-3018d5fe-2d82-4ca9-bc5d-537a3aa61114 admin admin] The token cache is being invalidate because project 87113ee29bca4ea7adb793bf89c4b392 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:53:53.283265 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-3018d5fe-2d82-4ca9-bc5d-537a3aa61114 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:53.284095 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-3018d5fe-2d82-4ca9-bc5d-537a3aa61114 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:53.285643 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 908/1821] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:53:52 2022] DELETE /identity/v3/projects/87113ee29bca4ea7adb793bf89c4b392 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:53.290596 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099474c8-8bf0-4733-b86d-46936f44dfdf None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:53.291035 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099474c8-8bf0-4733-b86d-46936f44dfdf None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:53.291317 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099474c8-8bf0-4733-b86d-46936f44dfdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:53.305981 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c042a7a5-508d-4836-b12c-3e432d898c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:53.306604 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042a7a5-508d-4836-b12c-3e432d898c17 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:53.306922 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042a7a5-508d-4836-b12c-3e432d898c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:53.307198 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042a7a5-508d-4836-b12c-3e432d898c17 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:53.307936 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c042a7a5-508d-4836-b12c-3e432d898c17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:53.308048 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-099474c8-8bf0-4733-b86d-46936f44dfdf None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:53.309190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-099474c8-8bf0-4733-b86d-46936f44dfdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:53.309564 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c042a7a5-508d-4836-b12c-3e432d898c17 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:53.317139 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 909/1822] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:53:53.322912 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 914/1823] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:53:53 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:53:53.324181 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099474c8-8bf0-4733-b86d-46936f44dfdf None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:53.325854 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-099474c8-8bf0-4733-b86d-46936f44dfdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:53.326145 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f54db3-326e-4089-a31e-3014cde0dff4 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:53.326497 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f54db3-326e-4089-a31e-3014cde0dff4 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:53.326899 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f54db3-326e-4089-a31e-3014cde0dff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:53.338389 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099474c8-8bf0-4733-b86d-46936f44dfdf None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:53.343327 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d5f936-af58-4d8b-b81a-ea34c5d89553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:53.344126 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d5f936-af58-4d8b-b81a-ea34c5d89553 admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:53.344407 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d5f936-af58-4d8b-b81a-ea34c5d89553 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:53.344677 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d5f936-af58-4d8b-b81a-ea34c5d89553 admin admin] PATH_INFO: `/v3/projects/8cf245b95d8b4061a01cb6dfb6dd767c/users/7849a9af90544991a0957ba4c714002f/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:53.345418 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-01f54db3-326e-4089-a31e-3014cde0dff4 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:53.346611 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-01f54db3-326e-4089-a31e-3014cde0dff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:53.346708 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d5f936-af58-4d8b-b81a-ea34c5d89553 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cf245b95d8b4061a01cb6dfb6dd767c, user_id=7849a9af90544991a0957ba4c714002f, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:53.349367 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d5f936-af58-4d8b-b81a-ea34c5d89553 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:53.354853 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 915/1824] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:53:53.358829 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f54db3-326e-4089-a31e-3014cde0dff4 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:53.360187 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 910/1825] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:53:53 2022] PUT /identity/v3/projects/8cf245b95d8b4061a01cb6dfb6dd767c/users/7849a9af90544991a0957ba4c714002f/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:53:53.367871 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f54db3-326e-4089-a31e-3014cde0dff4 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:53.377738 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442c900-0bc8-4106-bde5-142d54d6b382 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:53.377738 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442c900-0bc8-4106-bde5-142d54d6b382 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:53.377738 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442c900-0bc8-4106-bde5-142d54d6b382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:53.403134 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-2442c900-0bc8-4106-bde5-142d54d6b382 None None] MFA Rules not processed for user `7849a9af90544991a0957ba4c714002f`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:53.403134 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-2442c900-0bc8-4106-bde5-142d54d6b382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:53.440011 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6268a6c0-6082-4959-b8f7-e4d0f9fa5c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:53.440726 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268a6c0-6082-4959-b8f7-e4d0f9fa5c49 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:53.441034 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268a6c0-6082-4959-b8f7-e4d0f9fa5c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:53.441289 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268a6c0-6082-4959-b8f7-e4d0f9fa5c49 admin admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:53.442017 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6268a6c0-6082-4959-b8f7-e4d0f9fa5c49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:53:53.442325 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6268a6c0-6082-4959-b8f7-e4d0f9fa5c49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:53.444695 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6268a6c0-6082-4959-b8f7-e4d0f9fa5c49 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:53.457396 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 916/1826] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 22:53:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:53:53.461541 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6268a6c0-6082-4959-b8f7-e4d0f9fa5c49 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:53.472182 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6268a6c0-6082-4959-b8f7-e4d0f9fa5c49 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:53.481356 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 911/1827] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:53 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:53:53.484223 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a40d44-d09d-4efa-bb42-5c6849f16a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:53.484983 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a40d44-d09d-4efa-bb42-5c6849f16a10 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:53.485702 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a40d44-d09d-4efa-bb42-5c6849f16a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:53.485702 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a40d44-d09d-4efa-bb42-5c6849f16a10 admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:53.486426 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a40d44-d09d-4efa-bb42-5c6849f16a10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:53.488618 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a40d44-d09d-4efa-bb42-5c6849f16a10 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:53.497058 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2442c900-0bc8-4106-bde5-142d54d6b382 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:53.498452 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-2442c900-0bc8-4106-bde5-142d54d6b382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:53.557040 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2442c900-0bc8-4106-bde5-142d54d6b382 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:53.560728 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69312f7e-b72d-4991-a905-ba74dcd8e478 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:53.561473 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69312f7e-b72d-4991-a905-ba74dcd8e478 service cinder] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:53.561761 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69312f7e-b72d-4991-a905-ba74dcd8e478 service cinder] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:53.562039 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69312f7e-b72d-4991-a905-ba74dcd8e478 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:53.573231 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69312f7e-b72d-4991-a905-ba74dcd8e478 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:53.574712 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69312f7e-b72d-4991-a905-ba74dcd8e478 service cinder] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:53.582946 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 917/1828] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:53:53 2022] POST /identity/v3/projects => generated 407 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:53:53.582946 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a40d44-d09d-4efa-bb42-5c6849f16a10 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:53.589915 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 912/1829] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:53:53 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:53:53.644612 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69312f7e-b72d-4991-a905-ba74dcd8e478 service cinder] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:53.646451 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-69312f7e-b72d-4991-a905-ba74dcd8e478 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:53.656319 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a40d44-d09d-4efa-bb42-5c6849f16a10 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:53.657162 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69312f7e-b72d-4991-a905-ba74dcd8e478 service cinder] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:53.661439 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ff01a5-3a0d-439e-ad05-93c2829fcd06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:53.662121 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff01a5-3a0d-439e-ad05-93c2829fcd06 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:53.662410 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff01a5-3a0d-439e-ad05-93c2829fcd06 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:53.662720 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff01a5-3a0d-439e-ad05-93c2829fcd06 admin admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:53.663541 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ff01a5-3a0d-439e-ad05-93c2829fcd06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:53.665366 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ff01a5-3a0d-439e-ad05-93c2829fcd06 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:53.715971 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2547aada-2e74-4432-a1ea-623c331f7a57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:53.716729 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547aada-2e74-4432-a1ea-623c331f7a57 service cinder] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:53.716932 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547aada-2e74-4432-a1ea-623c331f7a57 service cinder] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:53.717214 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547aada-2e74-4432-a1ea-623c331f7a57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:53.719372 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-2547aada-2e74-4432-a1ea-623c331f7a57 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:53.766754 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 918/1830] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:53:53 2022] POST /identity/v3/users => generated 304 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:53:53.770860 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ff01a5-3a0d-439e-ad05-93c2829fcd06 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:53.784282 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ff01a5-3a0d-439e-ad05-93c2829fcd06 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:53.791118 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf53965-c62e-45b3-810e-d818accf21ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:53.792160 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf53965-c62e-45b3-810e-d818accf21ba admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:53.792543 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf53965-c62e-45b3-810e-d818accf21ba admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:53.792961 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf53965-c62e-45b3-810e-d818accf21ba admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:53.794492 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf53965-c62e-45b3-810e-d818accf21ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:53.796307 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf53965-c62e-45b3-810e-d818accf21ba admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:53.796751 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2547aada-2e74-4432-a1ea-623c331f7a57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:53.799019 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2547aada-2e74-4432-a1ea-623c331f7a57 service cinder] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:53.808428 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 919/1831] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:53:53 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:53:53.814819 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 913/1832] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:53:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:53:53.816430 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf53965-c62e-45b3-810e-d818accf21ba admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:53.831412 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf53965-c62e-45b3-810e-d818accf21ba admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:53.838468 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d860375-dd8c-438f-8f03-7c72b34fb551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:53.841100 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d860375-dd8c-438f-8f03-7c72b34fb551 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:53.841100 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d860375-dd8c-438f-8f03-7c72b34fb551 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:53.841100 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d860375-dd8c-438f-8f03-7c72b34fb551 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:53.841100 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d860375-dd8c-438f-8f03-7c72b34fb551 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:53.842216 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d860375-dd8c-438f-8f03-7c72b34fb551 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:53.853836 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 920/1833] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:53:53 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:53:53.858121 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2547aada-2e74-4432-a1ea-623c331f7a57 service cinder] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:53.869699 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2547aada-2e74-4432-a1ea-623c331f7a57 service cinder] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:53.874409 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a23e97b-c88d-4334-a424-6e03fd4ea0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:53.875367 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a23e97b-c88d-4334-a424-6e03fd4ea0db admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:53.875648 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a23e97b-c88d-4334-a424-6e03fd4ea0db admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:53.876650 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a23e97b-c88d-4334-a424-6e03fd4ea0db admin admin] PATH_INFO: `/v3/projects/13607936753648838ceb2807989a7dee/users/08a5e886e166462484d23ed97a9ec8ed/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:53.894082 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a23e97b-c88d-4334-a424-6e03fd4ea0db admin admin] RBAC: Authorizing `identity:create_grant(project_id=13607936753648838ceb2807989a7dee, user_id=08a5e886e166462484d23ed97a9ec8ed, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:53.898253 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a23e97b-c88d-4334-a424-6e03fd4ea0db admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:53.996947 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 914/1834] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:53:53 2022] PUT /identity/v3/projects/13607936753648838ceb2807989a7dee/users/08a5e886e166462484d23ed97a9ec8ed/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:54.012541 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc84d13-db76-4034-9eea-05782e13921d None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:54.013009 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc84d13-db76-4034-9eea-05782e13921d None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:54.013205 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc84d13-db76-4034-9eea-05782e13921d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:54.039740 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-5fc84d13-db76-4034-9eea-05782e13921d None None] MFA Rules not processed for user `08a5e886e166462484d23ed97a9ec8ed`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:54.041022 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-5fc84d13-db76-4034-9eea-05782e13921d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:54.048623 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=81,keystone:DELETE=16,keystone:INSERT=3 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:54.132506 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 921/1835] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:54 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:54.137227 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a23e97b-c88d-4334-a424-6e03fd4ea0db admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:54.230173 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a23e97b-c88d-4334-a424-6e03fd4ea0db admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:54.236833 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280edff1-6113-433c-b1a5-5c41c61b05de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:54.237667 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280edff1-6113-433c-b1a5-5c41c61b05de admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:54.238022 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280edff1-6113-433c-b1a5-5c41c61b05de admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:54.238447 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280edff1-6113-433c-b1a5-5c41c61b05de admin admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:54.239261 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280edff1-6113-433c-b1a5-5c41c61b05de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:54.241787 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280edff1-6113-433c-b1a5-5c41c61b05de admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:54.349005 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 915/1836] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:53:54 2022] POST /identity/v3/projects => generated 409 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:53:54.351025 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc84d13-db76-4034-9eea-05782e13921d None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:54.369273 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc84d13-db76-4034-9eea-05782e13921d None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:54.432636 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c750ba63-297a-4805-aaec-edcbf9564908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:54.433309 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750ba63-297a-4805-aaec-edcbf9564908 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:54.433620 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750ba63-297a-4805-aaec-edcbf9564908 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:54.433893 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750ba63-297a-4805-aaec-edcbf9564908 admin admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:54.434584 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c750ba63-297a-4805-aaec-edcbf9564908 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:54.436562 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c750ba63-297a-4805-aaec-edcbf9564908 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:54.564414 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 922/1837] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:53:54 2022] POST /identity/v3/users => generated 311 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:53:54.565884 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280edff1-6113-433c-b1a5-5c41c61b05de admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:54.591110 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280edff1-6113-433c-b1a5-5c41c61b05de admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:54.596232 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5bb371-3913-402b-aefc-f043d097190b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:54.598553 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5bb371-3913-402b-aefc-f043d097190b admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:54.598553 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5bb371-3913-402b-aefc-f043d097190b admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:54.598553 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5bb371-3913-402b-aefc-f043d097190b admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:54.598553 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5bb371-3913-402b-aefc-f043d097190b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:54.599866 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5bb371-3913-402b-aefc-f043d097190b admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:54.631979 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 916/1838] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:53:54 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:53:54.636364 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c750ba63-297a-4805-aaec-edcbf9564908 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:54.647797 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c750ba63-297a-4805-aaec-edcbf9564908 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:54.671685 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4128eaa1-1653-4f93-aa0e-534b617ef696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:54.678800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4128eaa1-1653-4f93-aa0e-534b617ef696 admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:54.679090 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4128eaa1-1653-4f93-aa0e-534b617ef696 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:54.679374 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4128eaa1-1653-4f93-aa0e-534b617ef696 admin admin] PATH_INFO: `/v3/projects/fab5e27ee808441b85c1f5ee94f585eb/users/ba7423531ad541edaea6f2e2942ec217/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:54.724536 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4128eaa1-1653-4f93-aa0e-534b617ef696 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fab5e27ee808441b85c1f5ee94f585eb, user_id=ba7423531ad541edaea6f2e2942ec217, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:54.727242 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4128eaa1-1653-4f93-aa0e-534b617ef696 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:54.841317 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 923/1839] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:53:54 2022] PUT /identity/v3/projects/fab5e27ee808441b85c1f5ee94f585eb/users/ba7423531ad541edaea6f2e2942ec217/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:53:54.842363 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5bb371-3913-402b-aefc-f043d097190b admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:54.862181 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5bb371-3913-402b-aefc-f043d097190b admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:54.978152 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bca3bc-6843-4e63-bf2a-d425814ad1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:54.978959 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bca3bc-6843-4e63-bf2a-d425814ad1b6 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:54.979310 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bca3bc-6843-4e63-bf2a-d425814ad1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:54.979671 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bca3bc-6843-4e63-bf2a-d425814ad1b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:54.980474 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bca3bc-6843-4e63-bf2a-d425814ad1b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:54.982205 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bca3bc-6843-4e63-bf2a-d425814ad1b6 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:54.994214 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 917/1840] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:53:54 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:53:54.998142 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4128eaa1-1653-4f93-aa0e-534b617ef696 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:55.024076 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4128eaa1-1653-4f93-aa0e-534b617ef696 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:55.039427 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a8fbb6-a494-4348-82e8-4cae320e4bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:55.040833 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8fbb6-a494-4348-82e8-4cae320e4bab admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:55.041232 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8fbb6-a494-4348-82e8-4cae320e4bab admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:55.041642 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8fbb6-a494-4348-82e8-4cae320e4bab admin admin] PATH_INFO: `/v3/projects/fab5e27ee808441b85c1f5ee94f585eb/users/ba7423531ad541edaea6f2e2942ec217/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:55.044181 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a8fbb6-a494-4348-82e8-4cae320e4bab admin admin] RBAC: Authorizing `identity:create_grant(project_id=fab5e27ee808441b85c1f5ee94f585eb, user_id=ba7423531ad541edaea6f2e2942ec217, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:55.054262 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a8fbb6-a494-4348-82e8-4cae320e4bab admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:55.071739 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 924/1841] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:53:54 2022] PUT /identity/v3/projects/fab5e27ee808441b85c1f5ee94f585eb/users/ba7423531ad541edaea6f2e2942ec217/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:55.089126 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc551b2-9512-4f40-8364-957d4ae850a5 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:55.089126 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc551b2-9512-4f40-8364-957d4ae850a5 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:55.089126 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc551b2-9512-4f40-8364-957d4ae850a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:55.155791 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-edc551b2-9512-4f40-8364-957d4ae850a5 None None] MFA Rules not processed for user `ba7423531ad541edaea6f2e2942ec217`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:55.157451 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-edc551b2-9512-4f40-8364-957d4ae850a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:55.279597 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 918/1842] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:55 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:55.292365 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a8fbb6-a494-4348-82e8-4cae320e4bab admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:55.376477 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a8fbb6-a494-4348-82e8-4cae320e4bab admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:55.381628 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bac80ce-e34e-43be-b97b-5f681159eac3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:55.382403 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac80ce-e34e-43be-b97b-5f681159eac3 service cinder] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:55.393591 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac80ce-e34e-43be-b97b-5f681159eac3 service cinder] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:55.393591 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac80ce-e34e-43be-b97b-5f681159eac3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:55.421691 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bac80ce-e34e-43be-b97b-5f681159eac3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:55.425270 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bac80ce-e34e-43be-b97b-5f681159eac3 service cinder] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:55.473067 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 925/1843] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:53:55 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:53:55.501349 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:56.007750 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc551b2-9512-4f40-8364-957d4ae850a5 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:56.018061 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc551b2-9512-4f40-8364-957d4ae850a5 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:56.022383 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83094924-83cd-41ea-b120-b84b952b874b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:56.023257 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83094924-83cd-41ea-b120-b84b952b874b service cinder] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:56.023767 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83094924-83cd-41ea-b120-b84b952b874b service cinder] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:56.024274 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83094924-83cd-41ea-b120-b84b952b874b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:56.116211 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83094924-83cd-41ea-b120-b84b952b874b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:56.117652 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83094924-83cd-41ea-b120-b84b952b874b service cinder] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:56.127260 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 919/1844] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:53:56 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:53:56.996738 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:57.325751 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:57.356382 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bac80ce-e34e-43be-b97b-5f681159eac3 service cinder] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:57.357851 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-1bac80ce-e34e-43be-b97b-5f681159eac3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:57.434708 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bac80ce-e34e-43be-b97b-5f681159eac3 service cinder] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:57.439360 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a860ac-f74a-470c-87ee-d091d46a681e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:57.440180 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a860ac-f74a-470c-87ee-d091d46a681e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:57.440465 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a860ac-f74a-470c-87ee-d091d46a681e admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:57.440738 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a860ac-f74a-470c-87ee-d091d46a681e admin admin] PATH_INFO: `/v3/users/f8ce9364111548dcab0d4bb28fd3a37d` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:57.441938 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a860ac-f74a-470c-87ee-d091d46a681e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8ce9364111548dcab0d4bb28fd3a37d)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:57.443905 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a860ac-f74a-470c-87ee-d091d46a681e admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:57.684123 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-74a860ac-f74a-470c-87ee-d091d46a681e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8ce9364111548dcab0d4bb28fd3a37d'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:57.703344 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-74a860ac-f74a-470c-87ee-d091d46a681e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8ce9364111548dcab0d4bb28fd3a37d'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:57.704115 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-74a860ac-f74a-470c-87ee-d091d46a681e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8ce9364111548dcab0d4bb28fd3a37d'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:57.732164 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-74a860ac-f74a-470c-87ee-d091d46a681e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8ce9364111548dcab0d4bb28fd3a37d'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:57.790260 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-74a860ac-f74a-470c-87ee-d091d46a681e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8ce9364111548dcab0d4bb28fd3a37d'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:57.816043 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 926/1845] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:53:57 2022] DELETE /identity/v3/users/f8ce9364111548dcab0d4bb28fd3a37d => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:57.820267 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f429062f-1d72-4d60-a038-24bc3cb6b1ee None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:57.820766 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f429062f-1d72-4d60-a038-24bc3cb6b1ee None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:57.820858 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f429062f-1d72-4d60-a038-24bc3cb6b1ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:57.856715 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-f429062f-1d72-4d60-a038-24bc3cb6b1ee None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:57.859594 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-f429062f-1d72-4d60-a038-24bc3cb6b1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:57.964252 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 920/1846] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:57.976060 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a860ac-f74a-470c-87ee-d091d46a681e admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:57.978436 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-74a860ac-f74a-470c-87ee-d091d46a681e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:57.987876 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bb9bff-2bab-4c7d-8d7a-7792feae1cb9 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:57.991613 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bb9bff-2bab-4c7d-8d7a-7792feae1cb9 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:57.991613 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bb9bff-2bab-4c7d-8d7a-7792feae1cb9 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:57.992700 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 921/1847] 10.222.0.52 () {54 vars in 921 bytes} [Thu Jul 21 22:53:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:53:58.032123 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb5c69-d2c5-4d93-966a-0d05cbc8cc5a None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:58.032403 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb5c69-d2c5-4d93-966a-0d05cbc8cc5a None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:58.032885 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb5c69-d2c5-4d93-966a-0d05cbc8cc5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:58.065422 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-03cb5c69-d2c5-4d93-966a-0d05cbc8cc5a None None] MFA Rules not processed for user `6bfdb655a2a44b33ab724daaf73a294b`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:58.066658 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-03cb5c69-d2c5-4d93-966a-0d05cbc8cc5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:58.069934 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a860ac-f74a-470c-87ee-d091d46a681e admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:58.076087 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf5ad74-d256-4f83-bae1-f5723dcddde2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:58.076955 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf5ad74-d256-4f83-bae1-f5723dcddde2 service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:58.079462 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf5ad74-d256-4f83-bae1-f5723dcddde2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:58.079906 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf5ad74-d256-4f83-bae1-f5723dcddde2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:58.092209 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf5ad74-d256-4f83-bae1-f5723dcddde2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:58.094652 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf5ad74-d256-4f83-bae1-f5723dcddde2 service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:58.106673 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 927/1848] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:53:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:53:58.212649 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 922/1849] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Jul 21 22:53:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:58.239778 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf5ad74-d256-4f83-bae1-f5723dcddde2 service neutron] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:58.241496 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-bcf5ad74-d256-4f83-bae1-f5723dcddde2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:58.270585 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf5ad74-d256-4f83-bae1-f5723dcddde2 service neutron] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:58.285433 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cff3d5-6b09-467c-a660-0e2ce64712e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bfdb655a2a44b33ab724daaf73a294b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:58.288171 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cff3d5-6b09-467c-a660-0e2ce64712e4 service nova] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:58.288332 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cff3d5-6b09-467c-a660-0e2ce64712e4 service nova] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:58.289058 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cff3d5-6b09-467c-a660-0e2ce64712e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:58.301969 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cff3d5-6b09-467c-a660-0e2ce64712e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:58.303975 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cff3d5-6b09-467c-a660-0e2ce64712e4 service nova] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:58.322158 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 928/1850] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:53:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:53:58.754596 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cb5c69-d2c5-4d93-966a-0d05cbc8cc5a None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:58.756144 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-03cb5c69-d2c5-4d93-966a-0d05cbc8cc5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:58.808311 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cb5c69-d2c5-4d93-966a-0d05cbc8cc5a None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:58.817157 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1705d11-712c-48b9-b5d5-9e0ec2a97f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:58.819050 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1705d11-712c-48b9-b5d5-9e0ec2a97f0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:58.819829 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1705d11-712c-48b9-b5d5-9e0ec2a97f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:58.820296 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1705d11-712c-48b9-b5d5-9e0ec2a97f0b admin admin] PATH_INFO: `/v3/projects/adf79b5233954a43a700b40b22b343b2` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:58.822538 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1705d11-712c-48b9-b5d5-9e0ec2a97f0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=adf79b5233954a43a700b40b22b343b2)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:58.828683 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1705d11-712c-48b9-b5d5-9e0ec2a97f0b admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:59.062938 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-d1705d11-712c-48b9-b5d5-9e0ec2a97f0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adf79b5233954a43a700b40b22b343b2', 'request_id': 'req-d1705d11-712c-48b9-b5d5-9e0ec2a97f0b'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:59.071001 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-d1705d11-712c-48b9-b5d5-9e0ec2a97f0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adf79b5233954a43a700b40b22b343b2', 'request_id': 'req-d1705d11-712c-48b9-b5d5-9e0ec2a97f0b'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:59.172699 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-d1705d11-712c-48b9-b5d5-9e0ec2a97f0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adf79b5233954a43a700b40b22b343b2', 'request_id': 'req-d1705d11-712c-48b9-b5d5-9e0ec2a97f0b'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:59.185239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-d1705d11-712c-48b9-b5d5-9e0ec2a97f0b admin admin] The token cache is being invalidate because project adf79b5233954a43a700b40b22b343b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:53:59.185665 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-d1705d11-712c-48b9-b5d5-9e0ec2a97f0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:59.186336 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-d1705d11-712c-48b9-b5d5-9e0ec2a97f0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:59.188281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 923/1851] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:53:58 2022] DELETE /identity/v3/projects/adf79b5233954a43a700b40b22b343b2 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:53:59.194958 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e09d7e-988f-4b53-b33a-066abe80a330 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:59.195462 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e09d7e-988f-4b53-b33a-066abe80a330 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:59.195541 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e09d7e-988f-4b53-b33a-066abe80a330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:59.213970 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-15e09d7e-988f-4b53-b33a-066abe80a330 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:59.215221 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-15e09d7e-988f-4b53-b33a-066abe80a330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:59.277302 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 929/1852] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:59.286554 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3688fa51-f56b-4948-a6c5-7806d0a19546 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:59.286859 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3688fa51-f56b-4948-a6c5-7806d0a19546 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:59.287179 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3688fa51-f56b-4948-a6c5-7806d0a19546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:59.307044 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-3688fa51-f56b-4948-a6c5-7806d0a19546 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:59.308295 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-3688fa51-f56b-4948-a6c5-7806d0a19546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:59.316565 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 924/1853] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:53:59.322132 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e09d7e-988f-4b53-b33a-066abe80a330 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:59.331909 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e09d7e-988f-4b53-b33a-066abe80a330 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:59.337199 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3b3be0-c5fe-4711-8d39-101a3022750b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:59.337873 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b3be0-c5fe-4711-8d39-101a3022750b admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:59.338162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b3be0-c5fe-4711-8d39-101a3022750b admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:59.338446 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b3be0-c5fe-4711-8d39-101a3022750b admin admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:59.339215 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3b3be0-c5fe-4711-8d39-101a3022750b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:53:59.339524 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3b3be0-c5fe-4711-8d39-101a3022750b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:59.341192 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3b3be0-c5fe-4711-8d39-101a3022750b admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:59.354078 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 930/1854] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 22:53:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:53:59.359315 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3688fa51-f56b-4948-a6c5-7806d0a19546 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:59.369110 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3688fa51-f56b-4948-a6c5-7806d0a19546 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:59.373901 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3e3c5-9720-48f3-9540-c0acc45d4dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:59.374621 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3e3c5-9720-48f3-9540-c0acc45d4dbd admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:59.374955 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3e3c5-9720-48f3-9540-c0acc45d4dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:59.376657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3e3c5-9720-48f3-9540-c0acc45d4dbd admin admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:59.376657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3e3c5-9720-48f3-9540-c0acc45d4dbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:59.377963 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3e3c5-9720-48f3-9540-c0acc45d4dbd admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:59.477410 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 925/1855] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:53:59 2022] POST /identity/v3/projects => generated 369 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:53:59.481990 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3b3be0-c5fe-4711-8d39-101a3022750b admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:59.494902 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3b3be0-c5fe-4711-8d39-101a3022750b admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:59.551683 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb6cb2f-d447-4f0f-aee6-98f2c04dbe73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:59.552808 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb6cb2f-d447-4f0f-aee6-98f2c04dbe73 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:59.553096 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb6cb2f-d447-4f0f-aee6-98f2c04dbe73 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:59.553376 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb6cb2f-d447-4f0f-aee6-98f2c04dbe73 admin admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:59.554075 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb6cb2f-d447-4f0f-aee6-98f2c04dbe73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:59.556028 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb6cb2f-d447-4f0f-aee6-98f2c04dbe73 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:59.631782 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 931/1856] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:53:59 2022] POST /identity/v3/users => generated 285 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:53:59.638410 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3e3c5-9720-48f3-9540-c0acc45d4dbd admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:59.649273 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3e3c5-9720-48f3-9540-c0acc45d4dbd admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:59.654257 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d03b56-d1d7-43b0-bbb0-df6656c6cac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:59.654989 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d03b56-d1d7-43b0-bbb0-df6656c6cac0 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:59.656046 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d03b56-d1d7-43b0-bbb0-df6656c6cac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:59.656046 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d03b56-d1d7-43b0-bbb0-df6656c6cac0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:59.656467 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d03b56-d1d7-43b0-bbb0-df6656c6cac0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:59.658017 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d03b56-d1d7-43b0-bbb0-df6656c6cac0 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:59.669263 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 926/1857] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:53:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:53:59.673803 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb6cb2f-d447-4f0f-aee6-98f2c04dbe73 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:59.686541 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb6cb2f-d447-4f0f-aee6-98f2c04dbe73 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:59.691005 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edb2707-81dd-4a6c-b9a3-d09540abf7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:59.691854 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb2707-81dd-4a6c-b9a3-d09540abf7d6 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:59.692146 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb2707-81dd-4a6c-b9a3-d09540abf7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:59.692458 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb2707-81dd-4a6c-b9a3-d09540abf7d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:59.693533 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d03b56-d1d7-43b0-bbb0-df6656c6cac0 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:59.693861 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edb2707-81dd-4a6c-b9a3-d09540abf7d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:59.695362 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-f2d03b56-d1d7-43b0-bbb0-df6656c6cac0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:59.695621 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edb2707-81dd-4a6c-b9a3-d09540abf7d6 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:59.707102 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 932/1858] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:53:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:53:59.711052 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edb2707-81dd-4a6c-b9a3-d09540abf7d6 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:59.712101 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d03b56-d1d7-43b0-bbb0-df6656c6cac0 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:59.717064 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1931e848-2a3d-40a7-9c10-7814ca1fa000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:59.718207 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931e848-2a3d-40a7-9c10-7814ca1fa000 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:59.718496 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931e848-2a3d-40a7-9c10-7814ca1fa000 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:59.718793 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931e848-2a3d-40a7-9c10-7814ca1fa000 admin admin] PATH_INFO: `/v3/users/0c1f5011fd3444b082b54fc1bd60dff3` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:59.727138 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1931e848-2a3d-40a7-9c10-7814ca1fa000 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c1f5011fd3444b082b54fc1bd60dff3)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:59.727138 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1931e848-2a3d-40a7-9c10-7814ca1fa000 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:59.727436 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edb2707-81dd-4a6c-b9a3-d09540abf7d6 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:53:59.727436 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a575a0-4281-44ad-bb1d-9e5ac0a764e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:53:59.728112 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a575a0-4281-44ad-bb1d-9e5ac0a764e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:59.728469 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a575a0-4281-44ad-bb1d-9e5ac0a764e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:59.728810 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a575a0-4281-44ad-bb1d-9e5ac0a764e1 admin admin] PATH_INFO: `/v3/projects/2e6a9a32d58d460c839eaeeabd890743/users/306adac7c3c3406fb5ab95d6d76eea9a/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:59.744995 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a575a0-4281-44ad-bb1d-9e5ac0a764e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e6a9a32d58d460c839eaeeabd890743, user_id=306adac7c3c3406fb5ab95d6d76eea9a, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:53:59.747774 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a575a0-4281-44ad-bb1d-9e5ac0a764e1 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:53:59.819153 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 933/1859] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:53:59 2022] PUT /identity/v3/projects/2e6a9a32d58d460c839eaeeabd890743/users/306adac7c3c3406fb5ab95d6d76eea9a/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:53:59.829425 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b6c4f5-4975-40c3-ac71-d7992ac30bce None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:53:59.829716 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b6c4f5-4975-40c3-ac71-d7992ac30bce None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:53:59.830000 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b6c4f5-4975-40c3-ac71-d7992ac30bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:53:59.856078 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-42b6c4f5-4975-40c3-ac71-d7992ac30bce None None] MFA Rules not processed for user `306adac7c3c3406fb5ab95d6d76eea9a`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:53:59.857295 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-42b6c4f5-4975-40c3-ac71-d7992ac30bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:59.859448 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:53:59.924877 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 934/1860] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:53:59 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:53:59.936885 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b6c4f5-4975-40c3-ac71-d7992ac30bce None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:53:59.941549 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-42b6c4f5-4975-40c3-ac71-d7992ac30bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:53:59.961636 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-1931e848-2a3d-40a7-9c10-7814ca1fa000 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c1f5011fd3444b082b54fc1bd60dff3'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:59.971104 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-1931e848-2a3d-40a7-9c10-7814ca1fa000 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c1f5011fd3444b082b54fc1bd60dff3'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:59.971833 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-1931e848-2a3d-40a7-9c10-7814ca1fa000 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c1f5011fd3444b082b54fc1bd60dff3'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:59.987404 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-1931e848-2a3d-40a7-9c10-7814ca1fa000 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c1f5011fd3444b082b54fc1bd60dff3'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:53:59.988217 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-1931e848-2a3d-40a7-9c10-7814ca1fa000 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c1f5011fd3444b082b54fc1bd60dff3'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:00.038663 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b6c4f5-4975-40c3-ac71-d7992ac30bce None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:00.042477 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fabd373-808a-402c-99cd-f593983b5c66 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '927778d005fd48a4b4b8fd3c4419bb36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:00.043168 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fabd373-808a-402c-99cd-f593983b5c66 service glance] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:00.043456 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fabd373-808a-402c-99cd-f593983b5c66 service glance] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:00.043734 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fabd373-808a-402c-99cd-f593983b5c66 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:00.045630 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-3fabd373-808a-402c-99cd-f593983b5c66 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:00.056942 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fabd373-808a-402c-99cd-f593983b5c66 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:00.058385 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fabd373-808a-402c-99cd-f593983b5c66 service glance] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:00.068600 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 935/1861] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:53:59 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:54:00.120704 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:00.126828 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 927/1862] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:53:59 2022] DELETE /identity/v3/users/0c1f5011fd3444b082b54fc1bd60dff3 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:00.135593 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91976135-2803-417d-b07f-18f9a26c6c17 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:00.135960 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91976135-2803-417d-b07f-18f9a26c6c17 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:00.136341 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91976135-2803-417d-b07f-18f9a26c6c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:00.155351 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-91976135-2803-417d-b07f-18f9a26c6c17 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:00.157152 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-91976135-2803-417d-b07f-18f9a26c6c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:00.292198 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 936/1863] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:54:00.304517 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1931e848-2a3d-40a7-9c10-7814ca1fa000 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:00.306329 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-1931e848-2a3d-40a7-9c10-7814ca1fa000 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:00.418814 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1931e848-2a3d-40a7-9c10-7814ca1fa000 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:00.423246 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3988f0a-ae81-4f3a-b08e-e3277bcd0be6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:00.424023 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3988f0a-ae81-4f3a-b08e-e3277bcd0be6 service neutron] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:00.424499 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3988f0a-ae81-4f3a-b08e-e3277bcd0be6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:00.424917 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3988f0a-ae81-4f3a-b08e-e3277bcd0be6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:00.437581 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3988f0a-ae81-4f3a-b08e-e3277bcd0be6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:00.439430 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3988f0a-ae81-4f3a-b08e-e3277bcd0be6 service neutron] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:00.451234 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 928/1864] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:54:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:54:00.901409 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91976135-2803-417d-b07f-18f9a26c6c17 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:00.903003 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-91976135-2803-417d-b07f-18f9a26c6c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:00.916154 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91976135-2803-417d-b07f-18f9a26c6c17 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:00.921827 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae31f84a-eac8-42ee-8556-3367cc0bf89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:00.922712 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31f84a-eac8-42ee-8556-3367cc0bf89f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:00.923060 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31f84a-eac8-42ee-8556-3367cc0bf89f admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:00.923339 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31f84a-eac8-42ee-8556-3367cc0bf89f admin admin] PATH_INFO: `/v3/projects/726799b752574673895da15b9b6b4aa7` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:00.924734 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae31f84a-eac8-42ee-8556-3367cc0bf89f admin admin] RBAC: Authorizing `identity:delete_project(project_id=726799b752574673895da15b9b6b4aa7)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:00.926679 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae31f84a-eac8-42ee-8556-3367cc0bf89f admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:01.044666 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-ae31f84a-eac8-42ee-8556-3367cc0bf89f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '726799b752574673895da15b9b6b4aa7', 'request_id': 'req-ae31f84a-eac8-42ee-8556-3367cc0bf89f'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:01.048579 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-ae31f84a-eac8-42ee-8556-3367cc0bf89f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '726799b752574673895da15b9b6b4aa7', 'request_id': 'req-ae31f84a-eac8-42ee-8556-3367cc0bf89f'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:01.119535 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-ae31f84a-eac8-42ee-8556-3367cc0bf89f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '726799b752574673895da15b9b6b4aa7', 'request_id': 'req-ae31f84a-eac8-42ee-8556-3367cc0bf89f'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:01.130894 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-ae31f84a-eac8-42ee-8556-3367cc0bf89f admin admin] The token cache is being invalidate because project 726799b752574673895da15b9b6b4aa7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:54:01.131199 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-ae31f84a-eac8-42ee-8556-3367cc0bf89f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:01.131831 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-ae31f84a-eac8-42ee-8556-3367cc0bf89f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:01.133384 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 937/1865] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:54:00 2022] DELETE /identity/v3/projects/726799b752574673895da15b9b6b4aa7 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:01.137392 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3988f0a-ae81-4f3a-b08e-e3277bcd0be6 service neutron] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:01.162454 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3988f0a-ae81-4f3a-b08e-e3277bcd0be6 service neutron] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:01.220936 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1655f45-277a-42b8-b5d0-9f14b9275298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:01.221880 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1655f45-277a-42b8-b5d0-9f14b9275298 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:01.222168 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1655f45-277a-42b8-b5d0-9f14b9275298 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:01.222449 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1655f45-277a-42b8-b5d0-9f14b9275298 admin admin] PATH_INFO: `/v3/users/7849a9af90544991a0957ba4c714002f` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:01.223734 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1655f45-277a-42b8-b5d0-9f14b9275298 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7849a9af90544991a0957ba4c714002f)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:01.226146 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1655f45-277a-42b8-b5d0-9f14b9275298 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:01.394720 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-f1655f45-277a-42b8-b5d0-9f14b9275298 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7849a9af90544991a0957ba4c714002f'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:01.406798 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-f1655f45-277a-42b8-b5d0-9f14b9275298 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7849a9af90544991a0957ba4c714002f'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:01.407855 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-f1655f45-277a-42b8-b5d0-9f14b9275298 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7849a9af90544991a0957ba4c714002f'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:01.422361 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-f1655f45-277a-42b8-b5d0-9f14b9275298 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7849a9af90544991a0957ba4c714002f'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:01.424192 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-f1655f45-277a-42b8-b5d0-9f14b9275298 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7849a9af90544991a0957ba4c714002f'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:01.505570 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 929/1866] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:54:01 2022] DELETE /identity/v3/users/7849a9af90544991a0957ba4c714002f => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:01.568154 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:01.704135 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:DELETE=17,keystone:SELECT=80,keystone:INSERT=3 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:01.923573 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae31f84a-eac8-42ee-8556-3367cc0bf89f admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:01.924485 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-ae31f84a-eac8-42ee-8556-3367cc0bf89f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:01.938618 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae31f84a-eac8-42ee-8556-3367cc0bf89f admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:01.999939 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53cf5ef-8513-450f-8b32-11d6eec138b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:02.001108 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53cf5ef-8513-450f-8b32-11d6eec138b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:02.001844 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53cf5ef-8513-450f-8b32-11d6eec138b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:02.002335 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53cf5ef-8513-450f-8b32-11d6eec138b1 admin admin] PATH_INFO: `/v3/projects/8cf245b95d8b4061a01cb6dfb6dd767c` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:02.003916 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53cf5ef-8513-450f-8b32-11d6eec138b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cf245b95d8b4061a01cb6dfb6dd767c)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:02.006705 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53cf5ef-8513-450f-8b32-11d6eec138b1 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:02.127130 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-c53cf5ef-8513-450f-8b32-11d6eec138b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cf245b95d8b4061a01cb6dfb6dd767c', 'request_id': 'req-c53cf5ef-8513-450f-8b32-11d6eec138b1'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:02.135606 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-c53cf5ef-8513-450f-8b32-11d6eec138b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cf245b95d8b4061a01cb6dfb6dd767c', 'request_id': 'req-c53cf5ef-8513-450f-8b32-11d6eec138b1'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:02.223999 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-c53cf5ef-8513-450f-8b32-11d6eec138b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cf245b95d8b4061a01cb6dfb6dd767c', 'request_id': 'req-c53cf5ef-8513-450f-8b32-11d6eec138b1'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:02.239772 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-c53cf5ef-8513-450f-8b32-11d6eec138b1 admin admin] The token cache is being invalidate because project 8cf245b95d8b4061a01cb6dfb6dd767c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:54:02.243349 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-c53cf5ef-8513-450f-8b32-11d6eec138b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:02.244267 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-c53cf5ef-8513-450f-8b32-11d6eec138b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:02.248448 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 938/1867] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:54:01 2022] DELETE /identity/v3/projects/8cf245b95d8b4061a01cb6dfb6dd767c => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:02.251299 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42804d51-994d-4337-954e-52e12619bb96 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:02.252126 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42804d51-994d-4337-954e-52e12619bb96 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:02.252481 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42804d51-994d-4337-954e-52e12619bb96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:02.280393 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-42804d51-994d-4337-954e-52e12619bb96 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:02.281855 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-42804d51-994d-4337-954e-52e12619bb96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:02.362484 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 930/1868] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:54:02.370490 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48f554-ab65-48b3-8014-618bfff7266f None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:02.370861 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48f554-ab65-48b3-8014-618bfff7266f None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:02.371147 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48f554-ab65-48b3-8014-618bfff7266f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:02.407256 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-7d48f554-ab65-48b3-8014-618bfff7266f None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:02.408491 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-7d48f554-ab65-48b3-8014-618bfff7266f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:02.417259 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 939/1869] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:54:02.421676 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42804d51-994d-4337-954e-52e12619bb96 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:02.432662 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42804d51-994d-4337-954e-52e12619bb96 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:02.437656 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e2cdd-4752-46da-9ad5-472759468ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:02.438342 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e2cdd-4752-46da-9ad5-472759468ce7 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:02.438627 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e2cdd-4752-46da-9ad5-472759468ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:02.438946 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e2cdd-4752-46da-9ad5-472759468ce7 admin admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:02.439750 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538e2cdd-4752-46da-9ad5-472759468ce7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:54:02.440064 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538e2cdd-4752-46da-9ad5-472759468ce7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:02.441676 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538e2cdd-4752-46da-9ad5-472759468ce7 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:02.454834 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 931/1870] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 22:54:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:54:02.459343 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d48f554-ab65-48b3-8014-618bfff7266f None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:02.473079 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d48f554-ab65-48b3-8014-618bfff7266f None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:02.505925 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e142ab-6653-4883-b748-d111d77f91e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:02.506665 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e142ab-6653-4883-b748-d111d77f91e5 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:02.507021 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e142ab-6653-4883-b748-d111d77f91e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:02.507302 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e142ab-6653-4883-b748-d111d77f91e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:02.508010 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e142ab-6653-4883-b748-d111d77f91e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:02.509925 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e142ab-6653-4883-b748-d111d77f91e5 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:02.617643 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 940/1871] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:54:02 2022] POST /identity/v3/projects => generated 401 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:54:02.621934 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e2cdd-4752-46da-9ad5-472759468ce7 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:02.633660 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e2cdd-4752-46da-9ad5-472759468ce7 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:02.694127 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e21b30-33ae-47c3-ad80-87db3b9a4805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:02.694835 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e21b30-33ae-47c3-ad80-87db3b9a4805 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:02.695129 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e21b30-33ae-47c3-ad80-87db3b9a4805 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:02.696265 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e21b30-33ae-47c3-ad80-87db3b9a4805 admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:02.697080 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e21b30-33ae-47c3-ad80-87db3b9a4805 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:02.697339 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:02.699066 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e21b30-33ae-47c3-ad80-87db3b9a4805 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:02.824621 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 932/1872] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:54:02 2022] POST /identity/v3/users => generated 301 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:54:02.829055 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e142ab-6653-4883-b748-d111d77f91e5 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:02.839780 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e142ab-6653-4883-b748-d111d77f91e5 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:02.846446 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612e785-1cad-4333-aaa5-6893892044b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:02.847186 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612e785-1cad-4333-aaa5-6893892044b7 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:02.847466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612e785-1cad-4333-aaa5-6893892044b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:02.847739 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612e785-1cad-4333-aaa5-6893892044b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:02.848466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7612e785-1cad-4333-aaa5-6893892044b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:02.850607 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7612e785-1cad-4333-aaa5-6893892044b7 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:02.862120 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 941/1873] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:54:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:54:02.867240 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e21b30-33ae-47c3-ad80-87db3b9a4805 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:02.877529 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e21b30-33ae-47c3-ad80-87db3b9a4805 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:02.882493 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8c7d57-8eac-4653-b2ed-96eea8f555e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:02.883317 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c7d57-8eac-4653-b2ed-96eea8f555e6 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:02.883603 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c7d57-8eac-4653-b2ed-96eea8f555e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:02.883875 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c7d57-8eac-4653-b2ed-96eea8f555e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:02.884599 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8c7d57-8eac-4653-b2ed-96eea8f555e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:02.886162 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8c7d57-8eac-4653-b2ed-96eea8f555e6 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:02.897900 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 933/1874] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:54:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:54:02.902183 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612e785-1cad-4333-aaa5-6893892044b7 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:02.913393 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612e785-1cad-4333-aaa5-6893892044b7 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:02.918019 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f00d1-e7f6-44f5-a34d-204687c10ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:02.918949 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f00d1-e7f6-44f5-a34d-204687c10ca3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:02.919240 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f00d1-e7f6-44f5-a34d-204687c10ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:02.919523 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f00d1-e7f6-44f5-a34d-204687c10ca3 admin admin] PATH_INFO: `/v3/projects/f79db8e9c70d44408b0376231cc6be39/users/a52070cfee9e4e66aec9b9885c2bc0a6/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:02.935267 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8f00d1-e7f6-44f5-a34d-204687c10ca3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f79db8e9c70d44408b0376231cc6be39, user_id=a52070cfee9e4e66aec9b9885c2bc0a6, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:02.937999 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8f00d1-e7f6-44f5-a34d-204687c10ca3 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:03.076830 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 942/1875] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:54:02 2022] PUT /identity/v3/projects/f79db8e9c70d44408b0376231cc6be39/users/a52070cfee9e4e66aec9b9885c2bc0a6/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:03.088430 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ef9c9-79d5-4fcb-ac96-fe15ed5856c6 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:03.088720 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ef9c9-79d5-4fcb-ac96-fe15ed5856c6 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:03.089222 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ef9c9-79d5-4fcb-ac96-fe15ed5856c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:03.120707 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-270ef9c9-79d5-4fcb-ac96-fe15ed5856c6 None None] MFA Rules not processed for user `a52070cfee9e4e66aec9b9885c2bc0a6`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:03.122147 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-270ef9c9-79d5-4fcb-ac96-fe15ed5856c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:03.193322 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 934/1876] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:03 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:54:03.197527 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f00d1-e7f6-44f5-a34d-204687c10ca3 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:03.288513 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f00d1-e7f6-44f5-a34d-204687c10ca3 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:03.298007 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636d3572-a824-4cfd-9cc3-1dee6faa965c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:03.298682 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636d3572-a824-4cfd-9cc3-1dee6faa965c admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:03.299009 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636d3572-a824-4cfd-9cc3-1dee6faa965c admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:03.299287 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636d3572-a824-4cfd-9cc3-1dee6faa965c admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:03.299991 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636d3572-a824-4cfd-9cc3-1dee6faa965c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:03.302040 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636d3572-a824-4cfd-9cc3-1dee6faa965c admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:03.427328 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 943/1877] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:54:03 2022] POST /identity/v3/projects => generated 401 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:54:03.431627 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270ef9c9-79d5-4fcb-ac96-fe15ed5856c6 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:03.459460 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270ef9c9-79d5-4fcb-ac96-fe15ed5856c6 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:03.548313 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0237b28b-d91e-4072-8a96-041131ae5b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:03.548910 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0237b28b-d91e-4072-8a96-041131ae5b57 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:03.549197 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0237b28b-d91e-4072-8a96-041131ae5b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:03.549467 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0237b28b-d91e-4072-8a96-041131ae5b57 admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:03.550155 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0237b28b-d91e-4072-8a96-041131ae5b57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:03.554342 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0237b28b-d91e-4072-8a96-041131ae5b57 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:03.681573 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:03.681835 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636d3572-a824-4cfd-9cc3-1dee6faa965c admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:03.689927 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 935/1878] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:54:03 2022] POST /identity/v3/users => generated 307 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:54:03.693339 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636d3572-a824-4cfd-9cc3-1dee6faa965c admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:03.698231 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c6819c-6751-4212-bc39-06849b2d47cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:03.698949 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6819c-6751-4212-bc39-06849b2d47cd admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:03.699240 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6819c-6751-4212-bc39-06849b2d47cd admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:03.699518 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6819c-6751-4212-bc39-06849b2d47cd admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:03.700290 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c6819c-6751-4212-bc39-06849b2d47cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:03.702803 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c6819c-6751-4212-bc39-06849b2d47cd admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:03.719771 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 944/1879] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:54:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:54:03.721503 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0237b28b-d91e-4072-8a96-041131ae5b57 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:03.732692 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0237b28b-d91e-4072-8a96-041131ae5b57 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:03.738156 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977eaf95-2696-4e56-9063-c7e4a59297ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:03.739202 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977eaf95-2696-4e56-9063-c7e4a59297ca admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:03.739490 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977eaf95-2696-4e56-9063-c7e4a59297ca admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:03.739809 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977eaf95-2696-4e56-9063-c7e4a59297ca admin admin] PATH_INFO: `/v3/projects/435dcbf817714def9485857ffaa88357/users/e7271dc331da44a3b026c58c6d9b5181/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:03.758433 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977eaf95-2696-4e56-9063-c7e4a59297ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=435dcbf817714def9485857ffaa88357, user_id=e7271dc331da44a3b026c58c6d9b5181, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:03.761324 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977eaf95-2696-4e56-9063-c7e4a59297ca admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:03.861781 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 936/1880] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:54:03 2022] PUT /identity/v3/projects/435dcbf817714def9485857ffaa88357/users/e7271dc331da44a3b026c58c6d9b5181/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:03.866745 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c6819c-6751-4212-bc39-06849b2d47cd admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:03.894053 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c6819c-6751-4212-bc39-06849b2d47cd admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:04.010761 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b43e679-323c-4887-b45c-75368064c643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:04.011856 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b43e679-323c-4887-b45c-75368064c643 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:04.012849 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b43e679-323c-4887-b45c-75368064c643 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:04.016692 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:04.016692 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b43e679-323c-4887-b45c-75368064c643 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:04.016692 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b43e679-323c-4887-b45c-75368064c643 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:04.017692 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b43e679-323c-4887-b45c-75368064c643 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:04.030812 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 945/1881] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:54:03 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:54:04.034848 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977eaf95-2696-4e56-9063-c7e4a59297ca admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:04.045957 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977eaf95-2696-4e56-9063-c7e4a59297ca admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:04.050709 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ec61ff-6a29-4bb9-b08f-d499f49a8797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:04.051616 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec61ff-6a29-4bb9-b08f-d499f49a8797 admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:04.051922 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec61ff-6a29-4bb9-b08f-d499f49a8797 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:04.052201 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec61ff-6a29-4bb9-b08f-d499f49a8797 admin admin] PATH_INFO: `/v3/projects/435dcbf817714def9485857ffaa88357/users/e7271dc331da44a3b026c58c6d9b5181/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:04.054269 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ec61ff-6a29-4bb9-b08f-d499f49a8797 admin admin] RBAC: Authorizing `identity:create_grant(project_id=435dcbf817714def9485857ffaa88357, user_id=e7271dc331da44a3b026c58c6d9b5181, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:04.057123 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ec61ff-6a29-4bb9-b08f-d499f49a8797 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:04.068281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 937/1882] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:54:04 2022] PUT /identity/v3/projects/435dcbf817714def9485857ffaa88357/users/e7271dc331da44a3b026c58c6d9b5181/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:04.078631 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf74d6-1154-4804-b76f-a8240537765e None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:04.078974 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf74d6-1154-4804-b76f-a8240537765e None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:04.079255 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf74d6-1154-4804-b76f-a8240537765e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:04.109380 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-8caf74d6-1154-4804-b76f-a8240537765e None None] MFA Rules not processed for user `e7271dc331da44a3b026c58c6d9b5181`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:04.110567 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-8caf74d6-1154-4804-b76f-a8240537765e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:04.178455 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 946/1883] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:04 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:54:04.189468 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ec61ff-6a29-4bb9-b08f-d499f49a8797 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:04.191224 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-98ec61ff-6a29-4bb9-b08f-d499f49a8797 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:04.253436 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ec61ff-6a29-4bb9-b08f-d499f49a8797 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:04.257113 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a019dbb-5490-40b1-958c-c99adfe94934 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:04.257779 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a019dbb-5490-40b1-958c-c99adfe94934 service cinder] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:04.263297 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a019dbb-5490-40b1-958c-c99adfe94934 service cinder] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:04.263297 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a019dbb-5490-40b1-958c-c99adfe94934 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:04.267473 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a019dbb-5490-40b1-958c-c99adfe94934 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:04.269119 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a019dbb-5490-40b1-958c-c99adfe94934 service cinder] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:04.284680 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 938/1884] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:54:04 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:54:04.523349 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf74d6-1154-4804-b76f-a8240537765e None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:04.533041 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf74d6-1154-4804-b76f-a8240537765e None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:04.537323 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bc8aff-8b71-4e83-9ec8-e6b1362d9b84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:04.538104 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc8aff-8b71-4e83-9ec8-e6b1362d9b84 service cinder] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:04.538248 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc8aff-8b71-4e83-9ec8-e6b1362d9b84 service cinder] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:04.538526 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc8aff-8b71-4e83-9ec8-e6b1362d9b84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:04.582475 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:04.600589 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bc8aff-8b71-4e83-9ec8-e6b1362d9b84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:04.602074 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bc8aff-8b71-4e83-9ec8-e6b1362d9b84 service cinder] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:04.611743 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 947/1885] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:54:04 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:54:04.698633 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:05.424481 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:05.761181 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:05.858277 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:06.224630 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:06.366966 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:06.435021 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:06.452181 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:06.547595 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:10.123949 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a019dbb-5490-40b1-958c-c99adfe94934 service cinder] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:10.125954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-7a019dbb-5490-40b1-958c-c99adfe94934 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:10.208238 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:10.232879 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a019dbb-5490-40b1-958c-c99adfe94934 service cinder] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:10.243464 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7171ed9-656f-40f5-be6b-ba7179a93298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:10.244589 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7171ed9-656f-40f5-be6b-ba7179a93298 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:10.244670 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7171ed9-656f-40f5-be6b-ba7179a93298 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:10.245043 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7171ed9-656f-40f5-be6b-ba7179a93298 admin admin] PATH_INFO: `/v3/users/08a5e886e166462484d23ed97a9ec8ed` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:10.246553 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7171ed9-656f-40f5-be6b-ba7179a93298 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08a5e886e166462484d23ed97a9ec8ed)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:10.252643 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7171ed9-656f-40f5-be6b-ba7179a93298 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:10.504995 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-f7171ed9-656f-40f5-be6b-ba7179a93298 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08a5e886e166462484d23ed97a9ec8ed'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:10.516337 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-f7171ed9-656f-40f5-be6b-ba7179a93298 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08a5e886e166462484d23ed97a9ec8ed'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:10.517949 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-f7171ed9-656f-40f5-be6b-ba7179a93298 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08a5e886e166462484d23ed97a9ec8ed'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:10.538312 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-f7171ed9-656f-40f5-be6b-ba7179a93298 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08a5e886e166462484d23ed97a9ec8ed'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:10.539084 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-f7171ed9-656f-40f5-be6b-ba7179a93298 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08a5e886e166462484d23ed97a9ec8ed'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:10.652832 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 939/1886] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:54:10 2022] DELETE /identity/v3/users/08a5e886e166462484d23ed97a9ec8ed => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:10.659384 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f5ebf-172e-4f26-a2f5-24af8680a590 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:10.659819 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f5ebf-172e-4f26-a2f5-24af8680a590 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:10.660092 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f5ebf-172e-4f26-a2f5-24af8680a590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:10.679962 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-031f5ebf-172e-4f26-a2f5-24af8680a590 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:10.681183 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-031f5ebf-172e-4f26-a2f5-24af8680a590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:10.748902 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:10.789395 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 948/1887] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:54:10.800367 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7171ed9-656f-40f5-be6b-ba7179a93298 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:10.801945 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-f7171ed9-656f-40f5-be6b-ba7179a93298 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:10.884412 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7171ed9-656f-40f5-be6b-ba7179a93298 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:10.893694 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c47d8d0-9ad8-465e-a799-888844945ac1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:10.901526 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47d8d0-9ad8-465e-a799-888844945ac1 service neutron] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:10.901526 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47d8d0-9ad8-465e-a799-888844945ac1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:10.901526 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47d8d0-9ad8-465e-a799-888844945ac1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:10.942122 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c47d8d0-9ad8-465e-a799-888844945ac1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:10.946334 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c47d8d0-9ad8-465e-a799-888844945ac1 service neutron] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:10.979298 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 940/1888] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:54:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:54:11.454584 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f5ebf-172e-4f26-a2f5-24af8680a590 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:11.456075 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-031f5ebf-172e-4f26-a2f5-24af8680a590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:11.468956 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f5ebf-172e-4f26-a2f5-24af8680a590 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:11.473933 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eaf68e-7bd6-4c19-a778-d0ddce5d03fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:11.475353 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eaf68e-7bd6-4c19-a778-d0ddce5d03fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:11.475878 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eaf68e-7bd6-4c19-a778-d0ddce5d03fb admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:11.475998 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eaf68e-7bd6-4c19-a778-d0ddce5d03fb admin admin] PATH_INFO: `/v3/projects/13607936753648838ceb2807989a7dee` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:11.477189 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eaf68e-7bd6-4c19-a778-d0ddce5d03fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=13607936753648838ceb2807989a7dee)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:11.479384 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eaf68e-7bd6-4c19-a778-d0ddce5d03fb admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:11.603055 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-f5eaf68e-7bd6-4c19-a778-d0ddce5d03fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13607936753648838ceb2807989a7dee', 'request_id': 'req-f5eaf68e-7bd6-4c19-a778-d0ddce5d03fb'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:11.610942 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-f5eaf68e-7bd6-4c19-a778-d0ddce5d03fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13607936753648838ceb2807989a7dee', 'request_id': 'req-f5eaf68e-7bd6-4c19-a778-d0ddce5d03fb'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:11.715161 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-f5eaf68e-7bd6-4c19-a778-d0ddce5d03fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13607936753648838ceb2807989a7dee', 'request_id': 'req-f5eaf68e-7bd6-4c19-a778-d0ddce5d03fb'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:11.730356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-f5eaf68e-7bd6-4c19-a778-d0ddce5d03fb admin admin] The token cache is being invalidate because project 13607936753648838ceb2807989a7dee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:54:11.731218 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-f5eaf68e-7bd6-4c19-a778-d0ddce5d03fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:11.731920 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-f5eaf68e-7bd6-4c19-a778-d0ddce5d03fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:11.733509 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 949/1889] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:54:11 2022] DELETE /identity/v3/projects/13607936753648838ceb2807989a7dee => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:11.737708 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c47d8d0-9ad8-465e-a799-888844945ac1 service neutron] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:11.748936 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c47d8d0-9ad8-465e-a799-888844945ac1 service neutron] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:11.813367 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f1aee0-a3ce-4132-ae8b-1f917711e666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:11.814183 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f1aee0-a3ce-4132-ae8b-1f917711e666 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:11.814475 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f1aee0-a3ce-4132-ae8b-1f917711e666 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:11.814730 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f1aee0-a3ce-4132-ae8b-1f917711e666 admin admin] PATH_INFO: `/v3/users/ba7423531ad541edaea6f2e2942ec217` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:11.815975 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f1aee0-a3ce-4132-ae8b-1f917711e666 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba7423531ad541edaea6f2e2942ec217)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:11.817995 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f1aee0-a3ce-4132-ae8b-1f917711e666 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:11.837575 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:12.035295 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-51f1aee0-a3ce-4132-ae8b-1f917711e666 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba7423531ad541edaea6f2e2942ec217'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:12.045093 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-51f1aee0-a3ce-4132-ae8b-1f917711e666 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba7423531ad541edaea6f2e2942ec217'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:12.046096 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-51f1aee0-a3ce-4132-ae8b-1f917711e666 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba7423531ad541edaea6f2e2942ec217'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:12.066498 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-51f1aee0-a3ce-4132-ae8b-1f917711e666 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba7423531ad541edaea6f2e2942ec217'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:12.067222 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-51f1aee0-a3ce-4132-ae8b-1f917711e666 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba7423531ad541edaea6f2e2942ec217'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:12.312748 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 941/1890] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:54:11 2022] DELETE /identity/v3/users/ba7423531ad541edaea6f2e2942ec217 => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:13.674302 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eaf68e-7bd6-4c19-a778-d0ddce5d03fb admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:13.679403 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-f5eaf68e-7bd6-4c19-a778-d0ddce5d03fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:13.680299 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f1aee0-a3ce-4132-ae8b-1f917711e666 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:13.681790 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-51f1aee0-a3ce-4132-ae8b-1f917711e666 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:13.695272 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eaf68e-7bd6-4c19-a778-d0ddce5d03fb admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:13.766547 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29092b61-5bd2-4126-b90c-b7135247a26b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:13.767682 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29092b61-5bd2-4126-b90c-b7135247a26b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:13.767980 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29092b61-5bd2-4126-b90c-b7135247a26b admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:13.768379 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29092b61-5bd2-4126-b90c-b7135247a26b admin admin] PATH_INFO: `/v3/users/a52070cfee9e4e66aec9b9885c2bc0a6` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:13.769571 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29092b61-5bd2-4126-b90c-b7135247a26b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a52070cfee9e4e66aec9b9885c2bc0a6)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:13.772084 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29092b61-5bd2-4126-b90c-b7135247a26b admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:13.787092 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f1aee0-a3ce-4132-ae8b-1f917711e666 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:13.792053 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0700ab-58de-4048-a289-e623e52674d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:13.792968 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0700ab-58de-4048-a289-e623e52674d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:13.793232 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0700ab-58de-4048-a289-e623e52674d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:13.793590 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0700ab-58de-4048-a289-e623e52674d5 admin admin] PATH_INFO: `/v3/users/d1aa30210cda4b6a9c457ba56f3a9e04` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:13.794837 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0700ab-58de-4048-a289-e623e52674d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1aa30210cda4b6a9c457ba56f3a9e04)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:13.796892 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0700ab-58de-4048-a289-e623e52674d5 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:13.952161 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-29092b61-5bd2-4126-b90c-b7135247a26b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a52070cfee9e4e66aec9b9885c2bc0a6'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:13.967241 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-29092b61-5bd2-4126-b90c-b7135247a26b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a52070cfee9e4e66aec9b9885c2bc0a6'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:13.971023 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-29092b61-5bd2-4126-b90c-b7135247a26b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a52070cfee9e4e66aec9b9885c2bc0a6'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:13.989639 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:13.991718 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-29092b61-5bd2-4126-b90c-b7135247a26b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a52070cfee9e4e66aec9b9885c2bc0a6'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:13.998832 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-3a0700ab-58de-4048-a289-e623e52674d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1aa30210cda4b6a9c457ba56f3a9e04'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:14.010401 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-3a0700ab-58de-4048-a289-e623e52674d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1aa30210cda4b6a9c457ba56f3a9e04'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:14.011156 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-3a0700ab-58de-4048-a289-e623e52674d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1aa30210cda4b6a9c457ba56f3a9e04'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:14.032331 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-3a0700ab-58de-4048-a289-e623e52674d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1aa30210cda4b6a9c457ba56f3a9e04'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:14.034197 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-3a0700ab-58de-4048-a289-e623e52674d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1aa30210cda4b6a9c457ba56f3a9e04'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:14.069505 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-29092b61-5bd2-4126-b90c-b7135247a26b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a52070cfee9e4e66aec9b9885c2bc0a6'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:14.076548 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 950/1891] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:54:13 2022] DELETE /identity/v3/users/a52070cfee9e4e66aec9b9885c2bc0a6 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:14.080288 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29092b61-5bd2-4126-b90c-b7135247a26b admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:14.080288 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-29092b61-5bd2-4126-b90c-b7135247a26b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:14.160937 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 942/1892] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:54:13 2022] DELETE /identity/v3/users/d1aa30210cda4b6a9c457ba56f3a9e04 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:14.160937 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af4cf2a-78ce-493f-831d-f68e15aab279 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:14.160937 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af4cf2a-78ce-493f-831d-f68e15aab279 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:14.160937 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af4cf2a-78ce-493f-831d-f68e15aab279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:14.186859 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29092b61-5bd2-4126-b90c-b7135247a26b admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:14.224489 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-8af4cf2a-78ce-493f-831d-f68e15aab279 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:14.224489 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-8af4cf2a-78ce-493f-831d-f68e15aab279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:14.231166 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:14.293847 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e1c45f-40ad-41c8-b384-979893a31a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:14.295037 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1c45f-40ad-41c8-b384-979893a31a91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:14.295325 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1c45f-40ad-41c8-b384-979893a31a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:14.295763 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1c45f-40ad-41c8-b384-979893a31a91 admin admin] PATH_INFO: `/v3/projects/fab5e27ee808441b85c1f5ee94f585eb` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:14.297420 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e1c45f-40ad-41c8-b384-979893a31a91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fab5e27ee808441b85c1f5ee94f585eb)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:14.300634 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e1c45f-40ad-41c8-b384-979893a31a91 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:14.325639 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 943/1893] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:54:14.326930 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecad542-24d7-4291-9b4c-16fd64d59de3 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:14.327225 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecad542-24d7-4291-9b4c-16fd64d59de3 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:14.328123 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecad542-24d7-4291-9b4c-16fd64d59de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:14.365437 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-1ecad542-24d7-4291-9b4c-16fd64d59de3 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:14.366923 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-1ecad542-24d7-4291-9b4c-16fd64d59de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:14.390995 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 944/1894] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:54:14.393624 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecad542-24d7-4291-9b4c-16fd64d59de3 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:14.396211 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-1ecad542-24d7-4291-9b4c-16fd64d59de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:14.458865 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-63e1c45f-40ad-41c8-b384-979893a31a91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fab5e27ee808441b85c1f5ee94f585eb', 'request_id': 'req-63e1c45f-40ad-41c8-b384-979893a31a91'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:14.472750 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-63e1c45f-40ad-41c8-b384-979893a31a91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fab5e27ee808441b85c1f5ee94f585eb', 'request_id': 'req-63e1c45f-40ad-41c8-b384-979893a31a91'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:14.548773 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecad542-24d7-4291-9b4c-16fd64d59de3 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:14.578007 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-63e1c45f-40ad-41c8-b384-979893a31a91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fab5e27ee808441b85c1f5ee94f585eb', 'request_id': 'req-63e1c45f-40ad-41c8-b384-979893a31a91'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:14.608333 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-63e1c45f-40ad-41c8-b384-979893a31a91 admin admin] The token cache is being invalidate because project fab5e27ee808441b85c1f5ee94f585eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:54:14.608637 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-63e1c45f-40ad-41c8-b384-979893a31a91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:14.609790 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-63e1c45f-40ad-41c8-b384-979893a31a91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:14.615870 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 951/1895] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:54:14 2022] DELETE /identity/v3/projects/fab5e27ee808441b85c1f5ee94f585eb => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:14.618506 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e1c45f-40ad-41c8-b384-979893a31a91 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:14.623487 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-63e1c45f-40ad-41c8-b384-979893a31a91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:14.661036 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d26cf28-e84e-41cf-81eb-abf4963634dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:14.667642 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d26cf28-e84e-41cf-81eb-abf4963634dd service neutron] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:14.667642 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d26cf28-e84e-41cf-81eb-abf4963634dd service neutron] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:14.667642 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d26cf28-e84e-41cf-81eb-abf4963634dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:14.667642 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-6d26cf28-e84e-41cf-81eb-abf4963634dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:14.701909 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:14.780923 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e1c45f-40ad-41c8-b384-979893a31a91 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:14.800603 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c086071f-22e0-4ae8-bab4-168128732a19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:14.801885 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d26cf28-e84e-41cf-81eb-abf4963634dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:14.803732 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c086071f-22e0-4ae8-bab4-168128732a19 service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:14.804849 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d26cf28-e84e-41cf-81eb-abf4963634dd service neutron] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:14.805308 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c086071f-22e0-4ae8-bab4-168128732a19 service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:14.805747 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c086071f-22e0-4ae8-bab4-168128732a19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:14.813621 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-c086071f-22e0-4ae8-bab4-168128732a19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:14.854741 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c086071f-22e0-4ae8-bab4-168128732a19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:14.856277 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c086071f-22e0-4ae8-bab4-168128732a19 service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:14.859442 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 945/1896] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:54:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:54:14.862810 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71086fc-4188-468d-8b2b-3d7fbe039b87 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:14.864085 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71086fc-4188-468d-8b2b-3d7fbe039b87 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:14.872387 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71086fc-4188-468d-8b2b-3d7fbe039b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:14.877883 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 952/1897] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:54:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:54:14.899120 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-a71086fc-4188-468d-8b2b-3d7fbe039b87 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:14.900751 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-a71086fc-4188-468d-8b2b-3d7fbe039b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:14.921510 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 946/1898] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:54:14.923523 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725ff1ef-f587-41ef-a003-cb4cd262d68d None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:14.923822 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725ff1ef-f587-41ef-a003-cb4cd262d68d None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:14.924108 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725ff1ef-f587-41ef-a003-cb4cd262d68d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:14.951079 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-725ff1ef-f587-41ef-a003-cb4cd262d68d None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:14.952386 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-725ff1ef-f587-41ef-a003-cb4cd262d68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:14.963255 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 953/1899] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:54:14.967037 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71086fc-4188-468d-8b2b-3d7fbe039b87 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:14.973437 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:14.983059 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71086fc-4188-468d-8b2b-3d7fbe039b87 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:14.993573 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a801c4c-04a0-4309-8fb4-e792dd5dfcce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:14.994969 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a801c4c-04a0-4309-8fb4-e792dd5dfcce admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:14.995412 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a801c4c-04a0-4309-8fb4-e792dd5dfcce admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:14.997218 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a801c4c-04a0-4309-8fb4-e792dd5dfcce admin admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:14.997218 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a801c4c-04a0-4309-8fb4-e792dd5dfcce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:54:14.997703 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a801c4c-04a0-4309-8fb4-e792dd5dfcce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:15.000957 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a801c4c-04a0-4309-8fb4-e792dd5dfcce admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:15.017987 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 947/1900] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 22:54:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:54:15.027649 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725ff1ef-f587-41ef-a003-cb4cd262d68d None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:15.039195 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725ff1ef-f587-41ef-a003-cb4cd262d68d None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:15.044326 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677ddc65-c96d-46fe-8462-65ae46fef0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:15.045127 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677ddc65-c96d-46fe-8462-65ae46fef0c4 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:15.045409 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677ddc65-c96d-46fe-8462-65ae46fef0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:15.045744 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677ddc65-c96d-46fe-8462-65ae46fef0c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:15.046506 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677ddc65-c96d-46fe-8462-65ae46fef0c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:15.052660 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677ddc65-c96d-46fe-8462-65ae46fef0c4 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:15.125205 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 954/1901] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:54:15 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:54:15.132446 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a801c4c-04a0-4309-8fb4-e792dd5dfcce admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:15.146091 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a801c4c-04a0-4309-8fb4-e792dd5dfcce admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:15.230636 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bda50ca-cb9c-4260-b565-baf45fc1adec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:15.231061 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda50ca-cb9c-4260-b565-baf45fc1adec admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:15.231344 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda50ca-cb9c-4260-b565-baf45fc1adec admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:15.231621 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda50ca-cb9c-4260-b565-baf45fc1adec admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:15.232444 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bda50ca-cb9c-4260-b565-baf45fc1adec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:15.234405 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bda50ca-cb9c-4260-b565-baf45fc1adec admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:15.327919 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 948/1902] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:54:15 2022] POST /identity/v3/users => generated 293 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:54:15.329672 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677ddc65-c96d-46fe-8462-65ae46fef0c4 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:15.348554 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677ddc65-c96d-46fe-8462-65ae46fef0c4 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:15.354954 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fdc68d-ad14-48f4-b427-853207c80ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:15.355746 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fdc68d-ad14-48f4-b427-853207c80ce2 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:15.356416 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fdc68d-ad14-48f4-b427-853207c80ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:15.356416 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fdc68d-ad14-48f4-b427-853207c80ce2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:15.357247 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fdc68d-ad14-48f4-b427-853207c80ce2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:15.358960 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fdc68d-ad14-48f4-b427-853207c80ce2 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:15.374368 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 955/1903] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:54:15 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:54:15.376100 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bda50ca-cb9c-4260-b565-baf45fc1adec admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:15.390018 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bda50ca-cb9c-4260-b565-baf45fc1adec admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:15.395446 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee38d61-9951-4a64-a34b-430230bd6915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:15.396136 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee38d61-9951-4a64-a34b-430230bd6915 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:15.396420 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee38d61-9951-4a64-a34b-430230bd6915 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:15.396695 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee38d61-9951-4a64-a34b-430230bd6915 admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:15.397429 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee38d61-9951-4a64-a34b-430230bd6915 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:15.399123 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee38d61-9951-4a64-a34b-430230bd6915 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:15.415282 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 949/1904] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:54:15 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:54:15.426100 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fdc68d-ad14-48f4-b427-853207c80ce2 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:15.435624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fdc68d-ad14-48f4-b427-853207c80ce2 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:15.441170 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecf50e5-a103-49ac-888b-ce1118bd6972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:15.442246 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecf50e5-a103-49ac-888b-ce1118bd6972 admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:15.443462 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecf50e5-a103-49ac-888b-ce1118bd6972 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:15.444559 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecf50e5-a103-49ac-888b-ce1118bd6972 admin admin] PATH_INFO: `/v3/projects/dc9e796be4b34c478ed35fcaec69147e/users/f3f61510ebe04e92967505678b0dc4e9/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:15.481664 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecf50e5-a103-49ac-888b-ce1118bd6972 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc9e796be4b34c478ed35fcaec69147e, user_id=f3f61510ebe04e92967505678b0dc4e9, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:15.492586 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecf50e5-a103-49ac-888b-ce1118bd6972 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:15.569517 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee38d61-9951-4a64-a34b-430230bd6915 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:15.572024 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-1ee38d61-9951-4a64-a34b-430230bd6915 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:15.590722 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 956/1905] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:54:15 2022] PUT /identity/v3/projects/dc9e796be4b34c478ed35fcaec69147e/users/f3f61510ebe04e92967505678b0dc4e9/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:15.593123 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3fe139-e54c-4515-9de1-e9f368f4de09 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:15.593485 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3fe139-e54c-4515-9de1-e9f368f4de09 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:15.593885 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3fe139-e54c-4515-9de1-e9f368f4de09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:15.604684 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee38d61-9951-4a64-a34b-430230bd6915 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:15.665052 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-7d3fe139-e54c-4515-9de1-e9f368f4de09 None None] MFA Rules not processed for user `f3f61510ebe04e92967505678b0dc4e9`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:15.666521 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-7d3fe139-e54c-4515-9de1-e9f368f4de09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:15.773776 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867674f-08bf-49e3-9f2c-a887f1a70cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:15.775191 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867674f-08bf-49e3-9f2c-a887f1a70cb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:15.775801 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867674f-08bf-49e3-9f2c-a887f1a70cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:15.775801 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867674f-08bf-49e3-9f2c-a887f1a70cb9 admin admin] PATH_INFO: `/v3/projects/f79db8e9c70d44408b0376231cc6be39` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:15.781704 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 957/1906] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:54:15.782183 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3867674f-08bf-49e3-9f2c-a887f1a70cb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f79db8e9c70d44408b0376231cc6be39)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:15.782294 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3fe139-e54c-4515-9de1-e9f368f4de09 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:15.783563 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-7d3fe139-e54c-4515-9de1-e9f368f4de09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:15.785519 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3867674f-08bf-49e3-9f2c-a887f1a70cb9 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:15.810964 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3fe139-e54c-4515-9de1-e9f368f4de09 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:15.823021 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2023ba6b-fec9-407a-9faf-d42f6ea62e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:15.824295 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2023ba6b-fec9-407a-9faf-d42f6ea62e94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:15.824590 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2023ba6b-fec9-407a-9faf-d42f6ea62e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:15.824868 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2023ba6b-fec9-407a-9faf-d42f6ea62e94 admin admin] PATH_INFO: `/v3/projects/8c0eb6e4060c4003b27ecf3401714f1d` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:15.826791 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2023ba6b-fec9-407a-9faf-d42f6ea62e94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c0eb6e4060c4003b27ecf3401714f1d)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:15.836810 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2023ba6b-fec9-407a-9faf-d42f6ea62e94 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:15.928685 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-3867674f-08bf-49e3-9f2c-a887f1a70cb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f79db8e9c70d44408b0376231cc6be39', 'request_id': 'req-3867674f-08bf-49e3-9f2c-a887f1a70cb9'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:15.953001 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-3867674f-08bf-49e3-9f2c-a887f1a70cb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f79db8e9c70d44408b0376231cc6be39', 'request_id': 'req-3867674f-08bf-49e3-9f2c-a887f1a70cb9'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:15.963066 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-2023ba6b-fec9-407a-9faf-d42f6ea62e94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c0eb6e4060c4003b27ecf3401714f1d', 'request_id': 'req-2023ba6b-fec9-407a-9faf-d42f6ea62e94'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:15.966581 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:15.973153 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-2023ba6b-fec9-407a-9faf-d42f6ea62e94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c0eb6e4060c4003b27ecf3401714f1d', 'request_id': 'req-2023ba6b-fec9-407a-9faf-d42f6ea62e94'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:16.035289 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-3867674f-08bf-49e3-9f2c-a887f1a70cb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f79db8e9c70d44408b0376231cc6be39', 'request_id': 'req-3867674f-08bf-49e3-9f2c-a887f1a70cb9'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:16.043599 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:16.048066 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-3867674f-08bf-49e3-9f2c-a887f1a70cb9 admin admin] The token cache is being invalidate because project f79db8e9c70d44408b0376231cc6be39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:54:16.048354 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-3867674f-08bf-49e3-9f2c-a887f1a70cb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:16.048997 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-3867674f-08bf-49e3-9f2c-a887f1a70cb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:16.050560 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 950/1907] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:54:15 2022] DELETE /identity/v3/projects/f79db8e9c70d44408b0376231cc6be39 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:16.051931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867674f-08bf-49e3-9f2c-a887f1a70cb9 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:16.053228 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-3867674f-08bf-49e3-9f2c-a887f1a70cb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:16.121800 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867674f-08bf-49e3-9f2c-a887f1a70cb9 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:16.122621 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-2023ba6b-fec9-407a-9faf-d42f6ea62e94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c0eb6e4060c4003b27ecf3401714f1d', 'request_id': 'req-2023ba6b-fec9-407a-9faf-d42f6ea62e94'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:16.143347 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-2023ba6b-fec9-407a-9faf-d42f6ea62e94 admin admin] The token cache is being invalidate because project 8c0eb6e4060c4003b27ecf3401714f1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:54:16.143692 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-2023ba6b-fec9-407a-9faf-d42f6ea62e94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:16.144674 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-2023ba6b-fec9-407a-9faf-d42f6ea62e94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:16.149654 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 958/1908] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:54:15 2022] DELETE /identity/v3/projects/8c0eb6e4060c4003b27ecf3401714f1d => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:16.152882 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2023ba6b-fec9-407a-9faf-d42f6ea62e94 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:16.155763 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-2023ba6b-fec9-407a-9faf-d42f6ea62e94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:16.177603 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85354cd-0913-44f0-adae-fbfd8ec37c2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:16.178536 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85354cd-0913-44f0-adae-fbfd8ec37c2f service cinder] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:16.180331 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85354cd-0913-44f0-adae-fbfd8ec37c2f service cinder] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:16.180331 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85354cd-0913-44f0-adae-fbfd8ec37c2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:16.181320 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-c85354cd-0913-44f0-adae-fbfd8ec37c2f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:16.265604 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2023ba6b-fec9-407a-9faf-d42f6ea62e94 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:16.273590 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8dfe8d-500c-4e51-b8c2-7d47e185355f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:16.274487 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8dfe8d-500c-4e51-b8c2-7d47e185355f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:16.274793 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8dfe8d-500c-4e51-b8c2-7d47e185355f admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:16.275076 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8dfe8d-500c-4e51-b8c2-7d47e185355f admin admin] PATH_INFO: `/v3/users/e7271dc331da44a3b026c58c6d9b5181` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:16.276299 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8dfe8d-500c-4e51-b8c2-7d47e185355f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7271dc331da44a3b026c58c6d9b5181)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:16.279653 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8dfe8d-500c-4e51-b8c2-7d47e185355f admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:16.281083 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85354cd-0913-44f0-adae-fbfd8ec37c2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:16.284006 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85354cd-0913-44f0-adae-fbfd8ec37c2f service cinder] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:16.295235 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 951/1909] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:54:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:54:16.295909 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85354cd-0913-44f0-adae-fbfd8ec37c2f service cinder] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:16.297317 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-c85354cd-0913-44f0-adae-fbfd8ec37c2f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:16.314041 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85354cd-0913-44f0-adae-fbfd8ec37c2f service cinder] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:16.322589 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaf773b-a156-46b8-8c8b-b332947abdff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:16.323717 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf773b-a156-46b8-8c8b-b332947abdff admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:16.323938 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf773b-a156-46b8-8c8b-b332947abdff admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:16.324271 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf773b-a156-46b8-8c8b-b332947abdff admin admin] PATH_INFO: `/v3/users/f547635351a04a62a97f196ec35e4d0b` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:16.325751 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaf773b-a156-46b8-8c8b-b332947abdff admin admin] RBAC: Authorizing `identity:delete_user(user_id=f547635351a04a62a97f196ec35e4d0b)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:16.327675 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaf773b-a156-46b8-8c8b-b332947abdff admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:16.546986 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-be8dfe8d-500c-4e51-b8c2-7d47e185355f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7271dc331da44a3b026c58c6d9b5181'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:16.557989 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-be8dfe8d-500c-4e51-b8c2-7d47e185355f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7271dc331da44a3b026c58c6d9b5181'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:16.558746 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-be8dfe8d-500c-4e51-b8c2-7d47e185355f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7271dc331da44a3b026c58c6d9b5181'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:16.569537 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-ceaf773b-a156-46b8-8c8b-b332947abdff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f547635351a04a62a97f196ec35e4d0b'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:16.572736 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-be8dfe8d-500c-4e51-b8c2-7d47e185355f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7271dc331da44a3b026c58c6d9b5181'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:16.577586 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-ceaf773b-a156-46b8-8c8b-b332947abdff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f547635351a04a62a97f196ec35e4d0b'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:16.578400 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-ceaf773b-a156-46b8-8c8b-b332947abdff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f547635351a04a62a97f196ec35e4d0b'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:16.591597 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-ceaf773b-a156-46b8-8c8b-b332947abdff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f547635351a04a62a97f196ec35e4d0b'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:16.592394 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-ceaf773b-a156-46b8-8c8b-b332947abdff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f547635351a04a62a97f196ec35e4d0b'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:16.610515 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:16.635079 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-be8dfe8d-500c-4e51-b8c2-7d47e185355f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7271dc331da44a3b026c58c6d9b5181'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:16.641871 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 959/1910] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:54:16 2022] DELETE /identity/v3/users/e7271dc331da44a3b026c58c6d9b5181 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:16.643025 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8dfe8d-500c-4e51-b8c2-7d47e185355f admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:16.645932 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-be8dfe8d-500c-4e51-b8c2-7d47e185355f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:16.712269 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 952/1911] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:54:16 2022] DELETE /identity/v3/users/f547635351a04a62a97f196ec35e4d0b => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:16.722725 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:16.727903 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8dfe8d-500c-4e51-b8c2-7d47e185355f admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:16.800692 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e76472a-2014-4f84-9014-c4392402f70a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:16.805894 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76472a-2014-4f84-9014-c4392402f70a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:16.805894 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76472a-2014-4f84-9014-c4392402f70a admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:16.805894 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76472a-2014-4f84-9014-c4392402f70a admin admin] PATH_INFO: `/v3/users/f3f61510ebe04e92967505678b0dc4e9` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:16.805894 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e76472a-2014-4f84-9014-c4392402f70a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3f61510ebe04e92967505678b0dc4e9)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:16.807366 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e76472a-2014-4f84-9014-c4392402f70a admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:17.063454 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-7e76472a-2014-4f84-9014-c4392402f70a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3f61510ebe04e92967505678b0dc4e9'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:17.076437 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-7e76472a-2014-4f84-9014-c4392402f70a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3f61510ebe04e92967505678b0dc4e9'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:17.077295 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-7e76472a-2014-4f84-9014-c4392402f70a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3f61510ebe04e92967505678b0dc4e9'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:17.093207 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-7e76472a-2014-4f84-9014-c4392402f70a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3f61510ebe04e92967505678b0dc4e9'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:17.169316 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaf773b-a156-46b8-8c8b-b332947abdff admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:17.172564 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-ceaf773b-a156-46b8-8c8b-b332947abdff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:17.181432 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-7e76472a-2014-4f84-9014-c4392402f70a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3f61510ebe04e92967505678b0dc4e9'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:17.193515 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 960/1912] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:54:16 2022] DELETE /identity/v3/users/f3f61510ebe04e92967505678b0dc4e9 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:17.195458 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e0198-f00d-4f04-a120-51e9bbfe7cae None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:17.196124 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e0198-f00d-4f04-a120-51e9bbfe7cae None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:17.196711 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e0198-f00d-4f04-a120-51e9bbfe7cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:17.224732 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaf773b-a156-46b8-8c8b-b332947abdff admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:17.290715 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-018e0198-f00d-4f04-a120-51e9bbfe7cae None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:17.292214 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-018e0198-f00d-4f04-a120-51e9bbfe7cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:17.341304 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e5dafc-b760-4cbd-8e32-442830683f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:17.342190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5dafc-b760-4cbd-8e32-442830683f86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:17.342724 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5dafc-b760-4cbd-8e32-442830683f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:17.343409 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e5dafc-b760-4cbd-8e32-442830683f86 admin admin] PATH_INFO: `/v3/projects/435dcbf817714def9485857ffaa88357` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:17.344707 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e5dafc-b760-4cbd-8e32-442830683f86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=435dcbf817714def9485857ffaa88357)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:17.347112 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e5dafc-b760-4cbd-8e32-442830683f86 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:17.403893 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 961/1913] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:54:17.414651 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018e0198-f00d-4f04-a120-51e9bbfe7cae None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:17.416208 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-018e0198-f00d-4f04-a120-51e9bbfe7cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:17.447050 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:17.511869 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018e0198-f00d-4f04-a120-51e9bbfe7cae None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:17.528104 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-75e5dafc-b760-4cbd-8e32-442830683f86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '435dcbf817714def9485857ffaa88357', 'request_id': 'req-75e5dafc-b760-4cbd-8e32-442830683f86'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:17.536766 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-75e5dafc-b760-4cbd-8e32-442830683f86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '435dcbf817714def9485857ffaa88357', 'request_id': 'req-75e5dafc-b760-4cbd-8e32-442830683f86'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:17.574157 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805a5a13-b802-423f-b630-0f5dfdc18760 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:17.575266 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a5a13-b802-423f-b630-0f5dfdc18760 service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:17.575740 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a5a13-b802-423f-b630-0f5dfdc18760 service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:17.576022 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a5a13-b802-423f-b630-0f5dfdc18760 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:17.600556 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-75e5dafc-b760-4cbd-8e32-442830683f86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '435dcbf817714def9485857ffaa88357', 'request_id': 'req-75e5dafc-b760-4cbd-8e32-442830683f86'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:17.614252 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-75e5dafc-b760-4cbd-8e32-442830683f86 admin admin] The token cache is being invalidate because project 435dcbf817714def9485857ffaa88357 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:54:17.614552 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-75e5dafc-b760-4cbd-8e32-442830683f86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:17.615239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-75e5dafc-b760-4cbd-8e32-442830683f86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:17.619957 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 953/1914] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:54:17 2022] DELETE /identity/v3/projects/435dcbf817714def9485857ffaa88357 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:17.619957 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e5dafc-b760-4cbd-8e32-442830683f86 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:17.620558 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-75e5dafc-b760-4cbd-8e32-442830683f86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:17.665805 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805a5a13-b802-423f-b630-0f5dfdc18760 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:17.667406 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805a5a13-b802-423f-b630-0f5dfdc18760 service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:17.684199 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 962/1915] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:54:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:54:17.686538 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e37c7f-b8b8-4b1f-a01c-72bc05537405 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:17.686978 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e37c7f-b8b8-4b1f-a01c-72bc05537405 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:17.687361 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e37c7f-b8b8-4b1f-a01c-72bc05537405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:17.712625 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e5dafc-b760-4cbd-8e32-442830683f86 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:17.719043 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-e1e37c7f-b8b8-4b1f-a01c-72bc05537405 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:17.720899 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-e1e37c7f-b8b8-4b1f-a01c-72bc05537405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:17.724988 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19731468-0d02-4681-a24a-386a3d4fc15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:17.725879 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19731468-0d02-4681-a24a-386a3d4fc15f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:17.726224 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19731468-0d02-4681-a24a-386a3d4fc15f admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:17.726509 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19731468-0d02-4681-a24a-386a3d4fc15f admin admin] PATH_INFO: `/v3/projects/cbdcb370ad4041aeaa7ce1f28ac041ed` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:17.729808 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19731468-0d02-4681-a24a-386a3d4fc15f admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbdcb370ad4041aeaa7ce1f28ac041ed)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:17.730211 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 963/1916] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:54:17.732083 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19731468-0d02-4681-a24a-386a3d4fc15f admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:17.737973 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16277de-c510-4be5-9155-12f5ec03fee4 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:17.738798 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16277de-c510-4be5-9155-12f5ec03fee4 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:17.739101 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16277de-c510-4be5-9155-12f5ec03fee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:17.751824 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:17.765318 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-c16277de-c510-4be5-9155-12f5ec03fee4 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:17.766885 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-c16277de-c510-4be5-9155-12f5ec03fee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:17.780436 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 964/1917] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:54:17.784514 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16277de-c510-4be5-9155-12f5ec03fee4 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:17.797869 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16277de-c510-4be5-9155-12f5ec03fee4 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:17.804461 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d30541-467a-480f-8991-d68031ffa709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:17.805313 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d30541-467a-480f-8991-d68031ffa709 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:17.805691 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d30541-467a-480f-8991-d68031ffa709 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:17.806139 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d30541-467a-480f-8991-d68031ffa709 admin admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:17.806944 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d30541-467a-480f-8991-d68031ffa709 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:54:17.807292 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d30541-467a-480f-8991-d68031ffa709 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:17.809667 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d30541-467a-480f-8991-d68031ffa709 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:17.824002 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 965/1918] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 22:54:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:54:17.828674 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d30541-467a-480f-8991-d68031ffa709 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:17.843214 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d30541-467a-480f-8991-d68031ffa709 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:17.859468 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe956393-30ca-47d7-91ce-d7eaa92a92ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:17.860354 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe956393-30ca-47d7-91ce-d7eaa92a92ff admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:17.860733 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe956393-30ca-47d7-91ce-d7eaa92a92ff admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:17.861112 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe956393-30ca-47d7-91ce-d7eaa92a92ff admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:17.862042 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe956393-30ca-47d7-91ce-d7eaa92a92ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:17.865350 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe956393-30ca-47d7-91ce-d7eaa92a92ff admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:17.915465 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-19731468-0d02-4681-a24a-386a3d4fc15f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbdcb370ad4041aeaa7ce1f28ac041ed', 'request_id': 'req-19731468-0d02-4681-a24a-386a3d4fc15f'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:17.922364 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-19731468-0d02-4681-a24a-386a3d4fc15f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbdcb370ad4041aeaa7ce1f28ac041ed', 'request_id': 'req-19731468-0d02-4681-a24a-386a3d4fc15f'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:17.994745 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 966/1919] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:54:17 2022] POST /identity/v3/projects => generated 367 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:54:17.995916 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe956393-30ca-47d7-91ce-d7eaa92a92ff admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:18.012552 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-19731468-0d02-4681-a24a-386a3d4fc15f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbdcb370ad4041aeaa7ce1f28ac041ed', 'request_id': 'req-19731468-0d02-4681-a24a-386a3d4fc15f'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:18.032970 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-19731468-0d02-4681-a24a-386a3d4fc15f admin admin] The token cache is being invalidate because project cbdcb370ad4041aeaa7ce1f28ac041ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:54:18.033333 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-19731468-0d02-4681-a24a-386a3d4fc15f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:18.035670 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-19731468-0d02-4681-a24a-386a3d4fc15f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:18.040294 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 954/1920] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:54:17 2022] DELETE /identity/v3/projects/cbdcb370ad4041aeaa7ce1f28ac041ed => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:18.045931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e958b2-4617-4536-9fb4-0b911e35cf76 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:18.046216 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e958b2-4617-4536-9fb4-0b911e35cf76 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:18.046548 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e958b2-4617-4536-9fb4-0b911e35cf76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:18.058837 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:18.079724 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe956393-30ca-47d7-91ce-d7eaa92a92ff admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:18.088174 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed296804-a091-4965-85e3-5302b666295d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:18.088869 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed296804-a091-4965-85e3-5302b666295d admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:18.089151 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed296804-a091-4965-85e3-5302b666295d admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:18.089714 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed296804-a091-4965-85e3-5302b666295d admin admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:18.090409 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed296804-a091-4965-85e3-5302b666295d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:18.092223 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-86e958b2-4617-4536-9fb4-0b911e35cf76 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:18.098762 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed296804-a091-4965-85e3-5302b666295d admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:18.098922 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-86e958b2-4617-4536-9fb4-0b911e35cf76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:18.115117 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 955/1921] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:54:18.132942 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c132af4-4b24-40d7-9ac5-03a9ba065a29 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:18.134154 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c132af4-4b24-40d7-9ac5-03a9ba065a29 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:18.134640 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c132af4-4b24-40d7-9ac5-03a9ba065a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:18.161316 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-9c132af4-4b24-40d7-9ac5-03a9ba065a29 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:18.162565 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-9c132af4-4b24-40d7-9ac5-03a9ba065a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:18.171639 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 956/1922] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:54:18.176932 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c132af4-4b24-40d7-9ac5-03a9ba065a29 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:18.188715 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c132af4-4b24-40d7-9ac5-03a9ba065a29 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:18.194334 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2cbc49-0e3c-4fae-bdaa-1899338b0f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:18.195052 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2cbc49-0e3c-4fae-bdaa-1899338b0f5a admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:18.195338 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2cbc49-0e3c-4fae-bdaa-1899338b0f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:18.195617 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2cbc49-0e3c-4fae-bdaa-1899338b0f5a admin admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:18.196356 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2cbc49-0e3c-4fae-bdaa-1899338b0f5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:54:18.196663 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2cbc49-0e3c-4fae-bdaa-1899338b0f5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:18.198321 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2cbc49-0e3c-4fae-bdaa-1899338b0f5a admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:18.215159 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 957/1923] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 22:54:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:54:18.218816 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 967/1924] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:54:17 2022] POST /identity/v3/users => generated 284 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:54:18.219857 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed296804-a091-4965-85e3-5302b666295d admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:18.222215 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-ed296804-a091-4965-85e3-5302b666295d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:18.225295 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2cbc49-0e3c-4fae-bdaa-1899338b0f5a admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:18.239933 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2cbc49-0e3c-4fae-bdaa-1899338b0f5a admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:18.243908 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed296804-a091-4965-85e3-5302b666295d admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:18.244839 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57af5571-1301-4efb-a3b8-a99bc29fadef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:18.245611 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af5571-1301-4efb-a3b8-a99bc29fadef admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:18.245944 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af5571-1301-4efb-a3b8-a99bc29fadef admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:18.246232 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af5571-1301-4efb-a3b8-a99bc29fadef admin admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:18.247161 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57af5571-1301-4efb-a3b8-a99bc29fadef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:18.249557 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57af5571-1301-4efb-a3b8-a99bc29fadef admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:18.251430 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb0eda-d6b5-437e-a850-65dafb5a64d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:18.252591 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb0eda-d6b5-437e-a850-65dafb5a64d0 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:18.252952 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb0eda-d6b5-437e-a850-65dafb5a64d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:18.253296 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb0eda-d6b5-437e-a850-65dafb5a64d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:18.254741 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddb0eda-d6b5-437e-a850-65dafb5a64d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:18.263773 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddb0eda-d6b5-437e-a850-65dafb5a64d0 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:18.276397 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 968/1925] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:54:18 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:54:18.280418 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb0eda-d6b5-437e-a850-65dafb5a64d0 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:18.291933 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb0eda-d6b5-437e-a850-65dafb5a64d0 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:18.296979 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846a2040-f31c-45cf-9433-411efeb9e5fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:18.297868 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a2040-f31c-45cf-9433-411efeb9e5fc admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:18.298220 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a2040-f31c-45cf-9433-411efeb9e5fc admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:18.298502 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a2040-f31c-45cf-9433-411efeb9e5fc admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:18.299276 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846a2040-f31c-45cf-9433-411efeb9e5fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:18.300914 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846a2040-f31c-45cf-9433-411efeb9e5fc admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:18.312902 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 969/1926] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:54:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:54:18.317165 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846a2040-f31c-45cf-9433-411efeb9e5fc admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:18.327613 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846a2040-f31c-45cf-9433-411efeb9e5fc admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:18.332937 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b1e40-c0cb-4d1f-b255-728d852f8104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:18.333847 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b1e40-c0cb-4d1f-b255-728d852f8104 admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:18.334157 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b1e40-c0cb-4d1f-b255-728d852f8104 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:18.334437 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b1e40-c0cb-4d1f-b255-728d852f8104 admin admin] PATH_INFO: `/v3/projects/c89813a1e3e04a3e87ba46cc1cd5dcc9/users/15ed73e2507244aab24d787b5b3b9794/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:18.354625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2b1e40-c0cb-4d1f-b255-728d852f8104 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c89813a1e3e04a3e87ba46cc1cd5dcc9, user_id=15ed73e2507244aab24d787b5b3b9794, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:18.358023 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2b1e40-c0cb-4d1f-b255-728d852f8104 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:18.361529 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 958/1927] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:54:18 2022] POST /identity/v3/projects => generated 379 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:54:18.361529 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57af5571-1301-4efb-a3b8-a99bc29fadef admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:18.363744 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-57af5571-1301-4efb-a3b8-a99bc29fadef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:18.433683 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57af5571-1301-4efb-a3b8-a99bc29fadef admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:18.438346 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2d863f-5c49-4889-a7a7-df41ca736578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:18.439260 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d863f-5c49-4889-a7a7-df41ca736578 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:18.439548 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d863f-5c49-4889-a7a7-df41ca736578 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:18.439826 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d863f-5c49-4889-a7a7-df41ca736578 admin admin] PATH_INFO: `/v3/projects/dc9e796be4b34c478ed35fcaec69147e` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:18.441044 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2d863f-5c49-4889-a7a7-df41ca736578 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc9e796be4b34c478ed35fcaec69147e)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:18.443171 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2d863f-5c49-4889-a7a7-df41ca736578 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:18.458877 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 970/1928] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:54:18 2022] PUT /identity/v3/projects/c89813a1e3e04a3e87ba46cc1cd5dcc9/users/15ed73e2507244aab24d787b5b3b9794/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:54:18.463760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b1e40-c0cb-4d1f-b255-728d852f8104 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:18.543355 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b1e40-c0cb-4d1f-b255-728d852f8104 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:18.549125 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1c2791-2428-496d-92cf-b99395408a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:18.553972 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c2791-2428-496d-92cf-b99395408a5c admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:18.553972 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c2791-2428-496d-92cf-b99395408a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:18.553972 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c2791-2428-496d-92cf-b99395408a5c admin admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:18.553972 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1c2791-2428-496d-92cf-b99395408a5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:18.553972 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1c2791-2428-496d-92cf-b99395408a5c admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:18.564030 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-ca2d863f-5c49-4889-a7a7-df41ca736578 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc9e796be4b34c478ed35fcaec69147e', 'request_id': 'req-ca2d863f-5c49-4889-a7a7-df41ca736578'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:18.569259 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:18.572013 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-ca2d863f-5c49-4889-a7a7-df41ca736578 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc9e796be4b34c478ed35fcaec69147e', 'request_id': 'req-ca2d863f-5c49-4889-a7a7-df41ca736578'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:18.626785 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 971/1929] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:54:18 2022] POST /identity/v3/users => generated 290 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:54:18.626785 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9283033-570e-4ccf-883d-2c341ddbf9b0 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:18.626785 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9283033-570e-4ccf-883d-2c341ddbf9b0 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:18.626785 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9283033-570e-4ccf-883d-2c341ddbf9b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:18.640771 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-ca2d863f-5c49-4889-a7a7-df41ca736578 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc9e796be4b34c478ed35fcaec69147e', 'request_id': 'req-ca2d863f-5c49-4889-a7a7-df41ca736578'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:18.653502 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-c9283033-570e-4ccf-883d-2c341ddbf9b0 None None] MFA Rules not processed for user `15ed73e2507244aab24d787b5b3b9794`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:18.654567 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-ca2d863f-5c49-4889-a7a7-df41ca736578 admin admin] The token cache is being invalidate because project dc9e796be4b34c478ed35fcaec69147e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:54:18.654765 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-c9283033-570e-4ccf-883d-2c341ddbf9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:18.654953 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-ca2d863f-5c49-4889-a7a7-df41ca736578 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:18.655584 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-ca2d863f-5c49-4889-a7a7-df41ca736578 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:18.657628 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 959/1930] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:54:18 2022] DELETE /identity/v3/projects/dc9e796be4b34c478ed35fcaec69147e => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:18.659302 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2d863f-5c49-4889-a7a7-df41ca736578 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:18.662204 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-ca2d863f-5c49-4889-a7a7-df41ca736578 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:18.722673 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 972/1931] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:18 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:54:18.724127 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584d4da-1502-4fd7-b65d-f05bea6aa6db None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:18.724509 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584d4da-1502-4fd7-b65d-f05bea6aa6db None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:18.725963 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584d4da-1502-4fd7-b65d-f05bea6aa6db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:18.733799 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2d863f-5c49-4889-a7a7-df41ca736578 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:18.738572 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fc8375-0b92-4f67-b62b-e6985b1a3262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:18.739276 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc8375-0b92-4f67-b62b-e6985b1a3262 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:18.739621 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc8375-0b92-4f67-b62b-e6985b1a3262 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:18.739838 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc8375-0b92-4f67-b62b-e6985b1a3262 admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:18.740571 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fc8375-0b92-4f67-b62b-e6985b1a3262 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:18.742329 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fc8375-0b92-4f67-b62b-e6985b1a3262 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:18.743284 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-8584d4da-1502-4fd7-b65d-f05bea6aa6db None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:18.744513 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-8584d4da-1502-4fd7-b65d-f05bea6aa6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:18.752625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 973/1932] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:54:18.752964 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 960/1933] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:54:18 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:54:18.753497 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8584d4da-1502-4fd7-b65d-f05bea6aa6db None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:18.756152 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:18.759301 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-8584d4da-1502-4fd7-b65d-f05bea6aa6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:18.766630 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeadab8-0dcb-404e-bd45-289c8f7b2f7d None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:18.767040 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeadab8-0dcb-404e-bd45-289c8f7b2f7d None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:18.767321 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeadab8-0dcb-404e-bd45-289c8f7b2f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:18.769268 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8584d4da-1502-4fd7-b65d-f05bea6aa6db None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:18.774493 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2dd4ec-c825-403f-b1fe-110b76bfd1ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:18.775199 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2dd4ec-c825-403f-b1fe-110b76bfd1ae admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:18.775489 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2dd4ec-c825-403f-b1fe-110b76bfd1ae admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:18.775916 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2dd4ec-c825-403f-b1fe-110b76bfd1ae admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:18.776616 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2dd4ec-c825-403f-b1fe-110b76bfd1ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:18.778530 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2dd4ec-c825-403f-b1fe-110b76bfd1ae admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:18.787291 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-ceeadab8-0dcb-404e-bd45-289c8f7b2f7d None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:18.788507 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-ceeadab8-0dcb-404e-bd45-289c8f7b2f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:18.797416 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 961/1934] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:54:18.798197 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeadab8-0dcb-404e-bd45-289c8f7b2f7d None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:18.807825 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeadab8-0dcb-404e-bd45-289c8f7b2f7d None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:18.812611 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8573eccc-3e12-4de1-8d56-a79092c8122c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:18.813244 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8573eccc-3e12-4de1-8d56-a79092c8122c admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:18.813527 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8573eccc-3e12-4de1-8d56-a79092c8122c admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:18.813804 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8573eccc-3e12-4de1-8d56-a79092c8122c admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:18.814515 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8573eccc-3e12-4de1-8d56-a79092c8122c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:18.816108 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8573eccc-3e12-4de1-8d56-a79092c8122c admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:18.827179 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 962/1935] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:54:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:54:18.828062 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8573eccc-3e12-4de1-8d56-a79092c8122c admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:18.837905 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8573eccc-3e12-4de1-8d56-a79092c8122c admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:18.843102 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651978fd-8a1b-4b83-9ba2-fcdc420e81af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:18.843754 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651978fd-8a1b-4b83-9ba2-fcdc420e81af admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:18.844036 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651978fd-8a1b-4b83-9ba2-fcdc420e81af admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:18.844314 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651978fd-8a1b-4b83-9ba2-fcdc420e81af admin admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:18.845037 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651978fd-8a1b-4b83-9ba2-fcdc420e81af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:54:18.845363 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651978fd-8a1b-4b83-9ba2-fcdc420e81af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:18.846970 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651978fd-8a1b-4b83-9ba2-fcdc420e81af admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:18.858829 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 963/1936] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 22:54:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:54:18.861030 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651978fd-8a1b-4b83-9ba2-fcdc420e81af admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:18.868604 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651978fd-8a1b-4b83-9ba2-fcdc420e81af admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:18.873096 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9026112f-7bb2-45ef-96f1-1ad54770a060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:18.873900 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9026112f-7bb2-45ef-96f1-1ad54770a060 admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:18.874204 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9026112f-7bb2-45ef-96f1-1ad54770a060 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:18.874437 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9026112f-7bb2-45ef-96f1-1ad54770a060 admin admin] PATH_INFO: `/v3/projects/7a8f1e318284416f96e8f609746cde82/users/b84b170c0bb3438885fe2ff890c5190b/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:18.889300 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9026112f-7bb2-45ef-96f1-1ad54770a060 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a8f1e318284416f96e8f609746cde82, user_id=b84b170c0bb3438885fe2ff890c5190b, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:18.891760 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9026112f-7bb2-45ef-96f1-1ad54770a060 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:18.920562 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 974/1937] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:54:18 2022] POST /identity/v3/projects => generated 365 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:54:18.920562 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2dd4ec-c825-403f-b1fe-110b76bfd1ae admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:18.973548 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 964/1938] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:54:18 2022] PUT /identity/v3/projects/7a8f1e318284416f96e8f609746cde82/users/b84b170c0bb3438885fe2ff890c5190b/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:18.974253 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9026112f-7bb2-45ef-96f1-1ad54770a060 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:18.983202 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2dd4ec-c825-403f-b1fe-110b76bfd1ae admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:19.040688 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a7998-5dbf-4069-9747-7e397cd0a5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:19.041381 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a7998-5dbf-4069-9747-7e397cd0a5f4 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:19.041665 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a7998-5dbf-4069-9747-7e397cd0a5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:19.042057 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a7998-5dbf-4069-9747-7e397cd0a5f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:19.042747 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183a7998-5dbf-4069-9747-7e397cd0a5f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:19.044624 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9026112f-7bb2-45ef-96f1-1ad54770a060 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:19.044723 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183a7998-5dbf-4069-9747-7e397cd0a5f4 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:19.049332 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea95ec-9f56-4c9e-b444-ca6e5b6a1e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:19.050001 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea95ec-9f56-4c9e-b444-ca6e5b6a1e05 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:19.050291 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea95ec-9f56-4c9e-b444-ca6e5b6a1e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:19.050566 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea95ec-9f56-4c9e-b444-ca6e5b6a1e05 admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:19.051824 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cea95ec-9f56-4c9e-b444-ca6e5b6a1e05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:19.053731 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cea95ec-9f56-4c9e-b444-ca6e5b6a1e05 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:19.128470 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 975/1939] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:54:18 2022] POST /identity/v3/projects => generated 375 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:54:19.128470 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbfd9ae-fc70-48e3-aba4-90df063077ce None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:19.128470 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbfd9ae-fc70-48e3-aba4-90df063077ce None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:19.128470 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbfd9ae-fc70-48e3-aba4-90df063077ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:19.154109 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-efbfd9ae-fc70-48e3-aba4-90df063077ce None None] MFA Rules not processed for user `b84b170c0bb3438885fe2ff890c5190b`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:19.155128 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 965/1940] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:54:18 2022] POST /identity/v3/users => generated 289 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:54:19.155506 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-efbfd9ae-fc70-48e3-aba4-90df063077ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:19.155951 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea95ec-9f56-4c9e-b444-ca6e5b6a1e05 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:19.166315 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea95ec-9f56-4c9e-b444-ca6e5b6a1e05 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:19.219032 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:DELETE=2,keystone:SELECT=93,keystone:INSERT=5 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:19.219703 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:19.228912 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 976/1941] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:19 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:54:19.228912 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbfd9ae-fc70-48e3-aba4-90df063077ce None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:19.229992 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645f64a1-328a-4220-b91c-ab802aa0ce63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:19.230858 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645f64a1-328a-4220-b91c-ab802aa0ce63 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:19.231316 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645f64a1-328a-4220-b91c-ab802aa0ce63 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:19.231763 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645f64a1-328a-4220-b91c-ab802aa0ce63 admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:19.232560 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645f64a1-328a-4220-b91c-ab802aa0ce63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:19.234829 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645f64a1-328a-4220-b91c-ab802aa0ce63 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:19.238070 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbfd9ae-fc70-48e3-aba4-90df063077ce None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:19.244283 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9182e08d-8b0a-48ac-a52b-abe62bf5a57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:19.244841 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182e08d-8b0a-48ac-a52b-abe62bf5a57e admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:19.245658 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182e08d-8b0a-48ac-a52b-abe62bf5a57e admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:19.246019 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182e08d-8b0a-48ac-a52b-abe62bf5a57e admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:19.246988 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9182e08d-8b0a-48ac-a52b-abe62bf5a57e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:19.249428 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9182e08d-8b0a-48ac-a52b-abe62bf5a57e admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:19.260714 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 977/1942] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:54:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:54:19.261685 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9182e08d-8b0a-48ac-a52b-abe62bf5a57e admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:19.271440 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9182e08d-8b0a-48ac-a52b-abe62bf5a57e admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:19.276290 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df88690-e7ae-4afa-badc-4e03a9265057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:19.276941 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df88690-e7ae-4afa-badc-4e03a9265057 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:19.277223 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df88690-e7ae-4afa-badc-4e03a9265057 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:19.277494 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df88690-e7ae-4afa-badc-4e03a9265057 admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:19.278181 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df88690-e7ae-4afa-badc-4e03a9265057 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:19.280113 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df88690-e7ae-4afa-badc-4e03a9265057 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:19.312281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 966/1943] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:54:19 2022] POST /identity/v3/users => generated 288 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:54:19.312281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645f64a1-328a-4220-b91c-ab802aa0ce63 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:19.322174 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645f64a1-328a-4220-b91c-ab802aa0ce63 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:19.327209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9602c22c-88f4-40ee-abb1-bf5ffcdd9258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:19.328127 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602c22c-88f4-40ee-abb1-bf5ffcdd9258 admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:19.328411 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602c22c-88f4-40ee-abb1-bf5ffcdd9258 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:19.328685 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602c22c-88f4-40ee-abb1-bf5ffcdd9258 admin admin] PATH_INFO: `/v3/projects/862b23fd95dd478aad7a4220a41ae8e6/users/a57abe9e71764d688150949759450b85/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:19.344541 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9602c22c-88f4-40ee-abb1-bf5ffcdd9258 admin admin] RBAC: Authorizing `identity:create_grant(project_id=862b23fd95dd478aad7a4220a41ae8e6, user_id=a57abe9e71764d688150949759450b85, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:19.347262 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9602c22c-88f4-40ee-abb1-bf5ffcdd9258 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:19.357558 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 978/1944] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:54:19 2022] POST /identity/v3/projects => generated 379 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:54:19.358595 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df88690-e7ae-4afa-badc-4e03a9265057 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:19.421196 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df88690-e7ae-4afa-badc-4e03a9265057 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:19.428186 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 967/1945] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:54:19 2022] PUT /identity/v3/projects/862b23fd95dd478aad7a4220a41ae8e6/users/a57abe9e71764d688150949759450b85/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:19.431663 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9602c22c-88f4-40ee-abb1-bf5ffcdd9258 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:19.484185 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1aa903-bc40-4e4e-9a5f-e53da51e477f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:19.484865 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1aa903-bc40-4e4e-9a5f-e53da51e477f admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:19.485151 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1aa903-bc40-4e4e-9a5f-e53da51e477f admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:19.485432 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1aa903-bc40-4e4e-9a5f-e53da51e477f admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:19.486239 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1aa903-bc40-4e4e-9a5f-e53da51e477f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:19.489086 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1aa903-bc40-4e4e-9a5f-e53da51e477f admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:19.494280 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9602c22c-88f4-40ee-abb1-bf5ffcdd9258 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:19.500323 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1df2dc2-6a2b-4c28-b833-98066aa8e689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:19.501004 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df2dc2-6a2b-4c28-b833-98066aa8e689 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:19.501290 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df2dc2-6a2b-4c28-b833-98066aa8e689 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:19.501569 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df2dc2-6a2b-4c28-b833-98066aa8e689 admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:19.502255 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1df2dc2-6a2b-4c28-b833-98066aa8e689 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:19.506125 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 979/1946] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:54:19 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:54:19.506279 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1df2dc2-6a2b-4c28-b833-98066aa8e689 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:19.506383 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1aa903-bc40-4e4e-9a5f-e53da51e477f admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:19.514366 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1aa903-bc40-4e4e-9a5f-e53da51e477f admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:19.518990 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6212d2f-0c1f-4880-876d-7dc22f2bb0fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:19.519651 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6212d2f-0c1f-4880-876d-7dc22f2bb0fa admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:19.519934 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6212d2f-0c1f-4880-876d-7dc22f2bb0fa admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:19.520206 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6212d2f-0c1f-4880-876d-7dc22f2bb0fa admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:19.520922 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6212d2f-0c1f-4880-876d-7dc22f2bb0fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:19.522604 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6212d2f-0c1f-4880-876d-7dc22f2bb0fa admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:19.533057 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 980/1947] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:54:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:54:19.533941 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6212d2f-0c1f-4880-876d-7dc22f2bb0fa admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:19.544038 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6212d2f-0c1f-4880-876d-7dc22f2bb0fa admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:19.548574 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d13a54b-93d3-4801-b995-f090eb10b5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:19.549200 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d13a54b-93d3-4801-b995-f090eb10b5d1 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:19.549481 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d13a54b-93d3-4801-b995-f090eb10b5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:19.549753 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d13a54b-93d3-4801-b995-f090eb10b5d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:19.550481 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d13a54b-93d3-4801-b995-f090eb10b5d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:19.552702 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d13a54b-93d3-4801-b995-f090eb10b5d1 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:19.562802 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 981/1948] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:54:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:54:19.563689 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d13a54b-93d3-4801-b995-f090eb10b5d1 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:19.572785 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d13a54b-93d3-4801-b995-f090eb10b5d1 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:19.578191 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2361526-abfd-4c96-9fbc-0231a8459c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:19.578191 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2361526-abfd-4c96-9fbc-0231a8459c3d admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:19.578399 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2361526-abfd-4c96-9fbc-0231a8459c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:19.578572 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2361526-abfd-4c96-9fbc-0231a8459c3d admin admin] PATH_INFO: `/v3/projects/862b23fd95dd478aad7a4220a41ae8e6/users/a57abe9e71764d688150949759450b85/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:19.580544 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2361526-abfd-4c96-9fbc-0231a8459c3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=862b23fd95dd478aad7a4220a41ae8e6, user_id=a57abe9e71764d688150949759450b85, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:19.583172 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2361526-abfd-4c96-9fbc-0231a8459c3d admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:19.593383 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 982/1949] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:54:19 2022] PUT /identity/v3/projects/862b23fd95dd478aad7a4220a41ae8e6/users/a57abe9e71764d688150949759450b85/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 30 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:19.594173 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2361526-abfd-4c96-9fbc-0231a8459c3d admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:19.611057 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 968/1950] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:54:19 2022] POST /identity/v3/users => generated 296 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:54:19.612367 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4dacd0-7bc3-4280-af39-b23c74ea2d39 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:19.612661 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4dacd0-7bc3-4280-af39-b23c74ea2d39 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:19.612942 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4dacd0-7bc3-4280-af39-b23c74ea2d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:19.628767 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:19.639808 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-7b4dacd0-7bc3-4280-af39-b23c74ea2d39 None None] MFA Rules not processed for user `a57abe9e71764d688150949759450b85`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:19.641178 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-7b4dacd0-7bc3-4280-af39-b23c74ea2d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:19.657011 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2361526-abfd-4c96-9fbc-0231a8459c3d admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:19.662513 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7800a9c-a8f3-4ba3-b27a-44cdb9619c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:19.663635 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7800a9c-a8f3-4ba3-b27a-44cdb9619c8f admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:19.663920 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7800a9c-a8f3-4ba3-b27a-44cdb9619c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:19.664196 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7800a9c-a8f3-4ba3-b27a-44cdb9619c8f admin admin] PATH_INFO: `/v3/projects/b9325bcb8a5c429383a0a51f086ec437/users/dd8483bf6f1044bc80b33892d7964e22/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:19.679169 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7800a9c-a8f3-4ba3-b27a-44cdb9619c8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9325bcb8a5c429383a0a51f086ec437, user_id=dd8483bf6f1044bc80b33892d7964e22, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:19.682271 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7800a9c-a8f3-4ba3-b27a-44cdb9619c8f admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:19.700960 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:19.711418 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 969/1951] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:19 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:54:19.712263 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4dacd0-7bc3-4280-af39-b23c74ea2d39 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:19.723381 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4dacd0-7bc3-4280-af39-b23c74ea2d39 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:19.728043 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83487bff-a3c4-4bdd-a94e-a7fc48145e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:19.728743 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83487bff-a3c4-4bdd-a94e-a7fc48145e9b admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:19.729033 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83487bff-a3c4-4bdd-a94e-a7fc48145e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:19.729314 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83487bff-a3c4-4bdd-a94e-a7fc48145e9b admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:19.730076 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83487bff-a3c4-4bdd-a94e-a7fc48145e9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:19.731711 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83487bff-a3c4-4bdd-a94e-a7fc48145e9b admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:19.737814 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 983/1952] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:54:19 2022] PUT /identity/v3/projects/b9325bcb8a5c429383a0a51f086ec437/users/dd8483bf6f1044bc80b33892d7964e22/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:19.738792 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7800a9c-a8f3-4ba3-b27a-44cdb9619c8f admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:19.740388 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-a7800a9c-a8f3-4ba3-b27a-44cdb9619c8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:19.742798 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 970/1953] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:54:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:54:19.748211 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2536d9-8825-4750-8230-53060af9c799 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:19.748507 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2536d9-8825-4750-8230-53060af9c799 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:19.748858 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2536d9-8825-4750-8230-53060af9c799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:19.775556 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-2c2536d9-8825-4750-8230-53060af9c799 None None] MFA Rules not processed for user `dd8483bf6f1044bc80b33892d7964e22`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:19.776785 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-2c2536d9-8825-4750-8230-53060af9c799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:19.800236 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7800a9c-a8f3-4ba3-b27a-44cdb9619c8f admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:19.805215 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c321c290-b695-4820-bfc2-af1cfc72e0ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:19.806219 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c321c290-b695-4820-bfc2-af1cfc72e0ec service cinder] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:19.806719 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c321c290-b695-4820-bfc2-af1cfc72e0ec service cinder] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:19.807192 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c321c290-b695-4820-bfc2-af1cfc72e0ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:19.850685 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 971/1954] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:54:19.850685 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2536d9-8825-4750-8230-53060af9c799 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:19.869141 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c321c290-b695-4820-bfc2-af1cfc72e0ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:19.870595 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c321c290-b695-4820-bfc2-af1cfc72e0ec service cinder] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:19.880308 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 984/1955] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:54:19 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:54:19.881178 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c321c290-b695-4820-bfc2-af1cfc72e0ec service cinder] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:19.882584 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-c321c290-b695-4820-bfc2-af1cfc72e0ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:19.913840 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2536d9-8825-4750-8230-53060af9c799 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:19.919117 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e814d39f-2c28-4d7d-b9e4-e7182bd622b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:19.920013 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e814d39f-2c28-4d7d-b9e4-e7182bd622b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:19.920298 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e814d39f-2c28-4d7d-b9e4-e7182bd622b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:19.920571 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e814d39f-2c28-4d7d-b9e4-e7182bd622b9 admin admin] PATH_INFO: `/v3/projects/51e60ebdb650445e9e3b996f846ef474/users/878e9ac29631475fb30ebac5b172b190/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:19.935370 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e814d39f-2c28-4d7d-b9e4-e7182bd622b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51e60ebdb650445e9e3b996f846ef474, user_id=878e9ac29631475fb30ebac5b172b190, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:19.938090 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e814d39f-2c28-4d7d-b9e4-e7182bd622b9 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:19.941587 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c321c290-b695-4820-bfc2-af1cfc72e0ec service cinder] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:19.945510 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8465a7a3-db08-45c2-878e-3e5fd488b75c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '927778d005fd48a4b4b8fd3c4419bb36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:19.946160 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8465a7a3-db08-45c2-878e-3e5fd488b75c service glance] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:19.946446 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8465a7a3-db08-45c2-878e-3e5fd488b75c service glance] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:19.946720 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8465a7a3-db08-45c2-878e-3e5fd488b75c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:19.955478 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8465a7a3-db08-45c2-878e-3e5fd488b75c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:19.957456 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8465a7a3-db08-45c2-878e-3e5fd488b75c service glance] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:19.967094 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 985/1956] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:54:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:54:20.028964 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 972/1957] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:54:19 2022] PUT /identity/v3/projects/51e60ebdb650445e9e3b996f846ef474/users/878e9ac29631475fb30ebac5b172b190/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:20.033096 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8465a7a3-db08-45c2-878e-3e5fd488b75c service glance] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:20.047696 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8465a7a3-db08-45c2-878e-3e5fd488b75c service glance] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:20.054103 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:20.110064 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96f4a48-28cb-409b-9a8f-7e05b9119737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:20.110744 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f4a48-28cb-409b-9a8f-7e05b9119737 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:20.111072 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f4a48-28cb-409b-9a8f-7e05b9119737 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:20.111351 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f4a48-28cb-409b-9a8f-7e05b9119737 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:20.112196 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96f4a48-28cb-409b-9a8f-7e05b9119737 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:20.113796 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96f4a48-28cb-409b-9a8f-7e05b9119737 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:20.123803 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 986/1958] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:54:20 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:54:20.128179 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e814d39f-2c28-4d7d-b9e4-e7182bd622b9 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:20.139934 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e814d39f-2c28-4d7d-b9e4-e7182bd622b9 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:20.145340 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b385a5-0f51-4217-9593-dd27551b8db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:20.146237 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b385a5-0f51-4217-9593-dd27551b8db2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:20.146522 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b385a5-0f51-4217-9593-dd27551b8db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:20.146924 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b385a5-0f51-4217-9593-dd27551b8db2 admin admin] PATH_INFO: `/v3/projects/51e60ebdb650445e9e3b996f846ef474/users/878e9ac29631475fb30ebac5b172b190/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:20.148033 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96f4a48-28cb-409b-9a8f-7e05b9119737 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:20.148940 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b385a5-0f51-4217-9593-dd27551b8db2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51e60ebdb650445e9e3b996f846ef474, user_id=878e9ac29631475fb30ebac5b172b190, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:20.151032 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-e96f4a48-28cb-409b-9a8f-7e05b9119737 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:20.152140 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b385a5-0f51-4217-9593-dd27551b8db2 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:20.164715 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 973/1959] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:54:20 2022] PUT /identity/v3/projects/51e60ebdb650445e9e3b996f846ef474/users/878e9ac29631475fb30ebac5b172b190/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:20.179338 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf525f-3ec5-42de-bd7a-e37cb010b83c None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:20.179635 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf525f-3ec5-42de-bd7a-e37cb010b83c None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:20.179909 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf525f-3ec5-42de-bd7a-e37cb010b83c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:20.211996 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-f2bf525f-3ec5-42de-bd7a-e37cb010b83c None None] MFA Rules not processed for user `878e9ac29631475fb30ebac5b172b190`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:20.213184 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-f2bf525f-3ec5-42de-bd7a-e37cb010b83c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:20.219577 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96f4a48-28cb-409b-9a8f-7e05b9119737 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:20.248820 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:20.275134 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe558a9-1081-4879-b013-5d518a2095e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:20.276152 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe558a9-1081-4879-b013-5d518a2095e5 service cinder] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:20.276444 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe558a9-1081-4879-b013-5d518a2095e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:20.276736 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe558a9-1081-4879-b013-5d518a2095e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:20.291842 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 974/1960] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:20 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:54:20.335049 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe558a9-1081-4879-b013-5d518a2095e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:20.336482 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe558a9-1081-4879-b013-5d518a2095e5 service cinder] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:20.345735 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 987/1961] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:54:20 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:54:20.797588 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bf525f-3ec5-42de-bd7a-e37cb010b83c None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:20.808815 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bf525f-3ec5-42de-bd7a-e37cb010b83c None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:20.812676 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1829ce-c6e6-4a75-9db9-6aaf11980fab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:20.813331 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1829ce-c6e6-4a75-9db9-6aaf11980fab service cinder] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:20.813618 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1829ce-c6e6-4a75-9db9-6aaf11980fab service cinder] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:20.813943 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1829ce-c6e6-4a75-9db9-6aaf11980fab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:20.823761 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1829ce-c6e6-4a75-9db9-6aaf11980fab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:20.825197 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1829ce-c6e6-4a75-9db9-6aaf11980fab service cinder] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:20.834847 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 975/1962] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:54:20 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:54:21.103032 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe558a9-1081-4879-b013-5d518a2095e5 service cinder] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:21.114233 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe558a9-1081-4879-b013-5d518a2095e5 service cinder] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:21.118723 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20051055-c83b-4c8b-bed5-468769bd0012 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:21.120648 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20051055-c83b-4c8b-bed5-468769bd0012 service cinder] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:21.121101 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20051055-c83b-4c8b-bed5-468769bd0012 service cinder] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:21.121553 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20051055-c83b-4c8b-bed5-468769bd0012 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:21.135127 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:21.181343 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20051055-c83b-4c8b-bed5-468769bd0012 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:21.183061 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20051055-c83b-4c8b-bed5-468769bd0012 service cinder] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:21.192069 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 988/1963] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:54:21 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:54:29.804071 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1829ce-c6e6-4a75-9db9-6aaf11980fab service cinder] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:29.876051 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1829ce-c6e6-4a75-9db9-6aaf11980fab service cinder] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:29.881490 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63506c36-ee2b-41f8-8933-9496a256f2b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:29.882817 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63506c36-ee2b-41f8-8933-9496a256f2b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:29.883105 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63506c36-ee2b-41f8-8933-9496a256f2b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:29.883391 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63506c36-ee2b-41f8-8933-9496a256f2b3 admin admin] PATH_INFO: `/v3/users/15ed73e2507244aab24d787b5b3b9794` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:29.884905 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63506c36-ee2b-41f8-8933-9496a256f2b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15ed73e2507244aab24d787b5b3b9794)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:29.887822 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63506c36-ee2b-41f8-8933-9496a256f2b3 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:30.039710 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-63506c36-ee2b-41f8-8933-9496a256f2b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15ed73e2507244aab24d787b5b3b9794'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:30.047955 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-63506c36-ee2b-41f8-8933-9496a256f2b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15ed73e2507244aab24d787b5b3b9794'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:30.048606 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-63506c36-ee2b-41f8-8933-9496a256f2b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15ed73e2507244aab24d787b5b3b9794'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:30.060776 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-63506c36-ee2b-41f8-8933-9496a256f2b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15ed73e2507244aab24d787b5b3b9794'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:30.061449 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-63506c36-ee2b-41f8-8933-9496a256f2b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15ed73e2507244aab24d787b5b3b9794'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:30.120120 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 976/1964] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:54:29 2022] DELETE /identity/v3/users/15ed73e2507244aab24d787b5b3b9794 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:30.124556 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54354689-13fe-4513-bf4e-5f121953fcd2 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:30.125045 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54354689-13fe-4513-bf4e-5f121953fcd2 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:30.125134 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54354689-13fe-4513-bf4e-5f121953fcd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:30.142318 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-54354689-13fe-4513-bf4e-5f121953fcd2 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:30.143555 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-54354689-13fe-4513-bf4e-5f121953fcd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:30.202385 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 989/1965] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:54:30.212522 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63506c36-ee2b-41f8-8933-9496a256f2b3 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:30.214204 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-63506c36-ee2b-41f8-8933-9496a256f2b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:30.249714 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:30.271182 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63506c36-ee2b-41f8-8933-9496a256f2b3 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:30.274842 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f483c1-ae4e-46eb-b888-55f48eb54697 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:30.275488 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f483c1-ae4e-46eb-b888-55f48eb54697 service neutron] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:30.275773 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f483c1-ae4e-46eb-b888-55f48eb54697 service neutron] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:30.276050 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f483c1-ae4e-46eb-b888-55f48eb54697 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:30.286381 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f483c1-ae4e-46eb-b888-55f48eb54697 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:30.287847 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f483c1-ae4e-46eb-b888-55f48eb54697 service neutron] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:30.297511 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 977/1966] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:54:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:54:30.704762 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54354689-13fe-4513-bf4e-5f121953fcd2 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:30.706957 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-54354689-13fe-4513-bf4e-5f121953fcd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:30.747211 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54354689-13fe-4513-bf4e-5f121953fcd2 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:30.751887 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bb9516-435e-4c9c-86aa-f2b43be7ad5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:30.752838 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb9516-435e-4c9c-86aa-f2b43be7ad5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:30.753127 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb9516-435e-4c9c-86aa-f2b43be7ad5f admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:30.753402 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb9516-435e-4c9c-86aa-f2b43be7ad5f admin admin] PATH_INFO: `/v3/projects/c89813a1e3e04a3e87ba46cc1cd5dcc9` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:30.755141 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bb9516-435e-4c9c-86aa-f2b43be7ad5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c89813a1e3e04a3e87ba46cc1cd5dcc9)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:30.757033 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bb9516-435e-4c9c-86aa-f2b43be7ad5f admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:30.866318 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-29bb9516-435e-4c9c-86aa-f2b43be7ad5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c89813a1e3e04a3e87ba46cc1cd5dcc9', 'request_id': 'req-29bb9516-435e-4c9c-86aa-f2b43be7ad5f'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:30.871812 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-29bb9516-435e-4c9c-86aa-f2b43be7ad5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c89813a1e3e04a3e87ba46cc1cd5dcc9', 'request_id': 'req-29bb9516-435e-4c9c-86aa-f2b43be7ad5f'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:30.935064 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-29bb9516-435e-4c9c-86aa-f2b43be7ad5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c89813a1e3e04a3e87ba46cc1cd5dcc9', 'request_id': 'req-29bb9516-435e-4c9c-86aa-f2b43be7ad5f'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:30.947615 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-29bb9516-435e-4c9c-86aa-f2b43be7ad5f admin admin] The token cache is being invalidate because project c89813a1e3e04a3e87ba46cc1cd5dcc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:54:30.947900 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-29bb9516-435e-4c9c-86aa-f2b43be7ad5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:30.948524 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-29bb9516-435e-4c9c-86aa-f2b43be7ad5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:30.950137 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 990/1967] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:54:30 2022] DELETE /identity/v3/projects/c89813a1e3e04a3e87ba46cc1cd5dcc9 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:30.954272 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f483c1-ae4e-46eb-b888-55f48eb54697 service neutron] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:30.964214 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f483c1-ae4e-46eb-b888-55f48eb54697 service neutron] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:31.023394 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef881ec-71e4-475c-8953-6a9689c7ce89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:31.027606 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef881ec-71e4-475c-8953-6a9689c7ce89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:31.027895 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef881ec-71e4-475c-8953-6a9689c7ce89 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:31.028172 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef881ec-71e4-475c-8953-6a9689c7ce89 admin admin] PATH_INFO: `/v3/users/a57abe9e71764d688150949759450b85` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:31.029584 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef881ec-71e4-475c-8953-6a9689c7ce89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a57abe9e71764d688150949759450b85)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:31.031538 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef881ec-71e4-475c-8953-6a9689c7ce89 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:31.279740 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-fef881ec-71e4-475c-8953-6a9689c7ce89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a57abe9e71764d688150949759450b85'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:31.287002 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-fef881ec-71e4-475c-8953-6a9689c7ce89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a57abe9e71764d688150949759450b85'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:31.287662 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-fef881ec-71e4-475c-8953-6a9689c7ce89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a57abe9e71764d688150949759450b85'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:31.299408 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-fef881ec-71e4-475c-8953-6a9689c7ce89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a57abe9e71764d688150949759450b85'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:31.304934 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-fef881ec-71e4-475c-8953-6a9689c7ce89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a57abe9e71764d688150949759450b85'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:31.374483 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 978/1968] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:54:30 2022] DELETE /identity/v3/users/a57abe9e71764d688150949759450b85 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:31.774455 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bb9516-435e-4c9c-86aa-f2b43be7ad5f admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:31.776007 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-29bb9516-435e-4c9c-86aa-f2b43be7ad5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:31.790939 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bb9516-435e-4c9c-86aa-f2b43be7ad5f admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:31.851324 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:31.891292 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16744729-dac2-48bd-a7ec-8ae898fad41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:31.895739 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16744729-dac2-48bd-a7ec-8ae898fad41e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:31.895739 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16744729-dac2-48bd-a7ec-8ae898fad41e admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:31.895739 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16744729-dac2-48bd-a7ec-8ae898fad41e admin admin] PATH_INFO: `/v3/projects/862b23fd95dd478aad7a4220a41ae8e6` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:31.895739 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16744729-dac2-48bd-a7ec-8ae898fad41e admin admin] RBAC: Authorizing `identity:delete_project(project_id=862b23fd95dd478aad7a4220a41ae8e6)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:31.897207 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16744729-dac2-48bd-a7ec-8ae898fad41e admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:32.028804 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-16744729-dac2-48bd-a7ec-8ae898fad41e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '862b23fd95dd478aad7a4220a41ae8e6', 'request_id': 'req-16744729-dac2-48bd-a7ec-8ae898fad41e'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:32.035769 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-16744729-dac2-48bd-a7ec-8ae898fad41e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '862b23fd95dd478aad7a4220a41ae8e6', 'request_id': 'req-16744729-dac2-48bd-a7ec-8ae898fad41e'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:32.114201 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-16744729-dac2-48bd-a7ec-8ae898fad41e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '862b23fd95dd478aad7a4220a41ae8e6', 'request_id': 'req-16744729-dac2-48bd-a7ec-8ae898fad41e'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:32.126630 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-16744729-dac2-48bd-a7ec-8ae898fad41e admin admin] The token cache is being invalidate because project 862b23fd95dd478aad7a4220a41ae8e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:54:32.126955 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-16744729-dac2-48bd-a7ec-8ae898fad41e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:32.127600 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-16744729-dac2-48bd-a7ec-8ae898fad41e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:32.130257 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 991/1969] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:54:31 2022] DELETE /identity/v3/projects/862b23fd95dd478aad7a4220a41ae8e6 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:32.135001 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d5dee-0c9a-4133-ad02-794c8abbb903 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:32.135528 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d5dee-0c9a-4133-ad02-794c8abbb903 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:32.136159 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d5dee-0c9a-4133-ad02-794c8abbb903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:32.156558 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-fb6d5dee-0c9a-4133-ad02-794c8abbb903 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:32.157765 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-fb6d5dee-0c9a-4133-ad02-794c8abbb903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:32.218939 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:32.230377 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 979/1970] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:54:32.239333 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2fbb4-31ba-4064-a3cc-3c5557f509c3 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:32.239702 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2fbb4-31ba-4064-a3cc-3c5557f509c3 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:32.240048 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2fbb4-31ba-4064-a3cc-3c5557f509c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:32.262379 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-6de2fbb4-31ba-4064-a3cc-3c5557f509c3 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:32.263453 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-6de2fbb4-31ba-4064-a3cc-3c5557f509c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:32.276000 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 992/1971] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:54:32.286219 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6d5dee-0c9a-4133-ad02-794c8abbb903 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:32.296134 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6d5dee-0c9a-4133-ad02-794c8abbb903 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:32.300715 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce15d36-bb08-436f-9e86-e8191c9324d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:32.301498 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce15d36-bb08-436f-9e86-e8191c9324d4 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:32.301787 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce15d36-bb08-436f-9e86-e8191c9324d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:32.302075 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce15d36-bb08-436f-9e86-e8191c9324d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:32.302804 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce15d36-bb08-436f-9e86-e8191c9324d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:54:32.303182 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce15d36-bb08-436f-9e86-e8191c9324d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:32.307294 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce15d36-bb08-436f-9e86-e8191c9324d4 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:32.321771 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 980/1972] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 22:54:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:54:32.326765 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de2fbb4-31ba-4064-a3cc-3c5557f509c3 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:32.338399 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de2fbb4-31ba-4064-a3cc-3c5557f509c3 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:32.350158 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa373c-f858-4337-acdf-9a046f8ce7c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:32.350158 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa373c-f858-4337-acdf-9a046f8ce7c4 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:32.350158 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa373c-f858-4337-acdf-9a046f8ce7c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:32.350158 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa373c-f858-4337-acdf-9a046f8ce7c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:32.350577 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa373c-f858-4337-acdf-9a046f8ce7c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:32.350577 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa373c-f858-4337-acdf-9a046f8ce7c4 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:32.455800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 993/1973] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:54:32 2022] POST /identity/v3/projects => generated 375 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:54:32.457008 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce15d36-bb08-436f-9e86-e8191c9324d4 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:32.469703 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce15d36-bb08-436f-9e86-e8191c9324d4 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:32.566000 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f057bb-6a9b-42a7-a6bc-a544db43268e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:32.566682 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f057bb-6a9b-42a7-a6bc-a544db43268e admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:32.567568 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f057bb-6a9b-42a7-a6bc-a544db43268e admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:32.567844 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f057bb-6a9b-42a7-a6bc-a544db43268e admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:32.568535 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f057bb-6a9b-42a7-a6bc-a544db43268e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:32.570537 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f057bb-6a9b-42a7-a6bc-a544db43268e admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:32.657300 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 981/1974] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:54:32 2022] POST /identity/v3/users => generated 288 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:54:32.661498 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa373c-f858-4337-acdf-9a046f8ce7c4 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:32.673458 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa373c-f858-4337-acdf-9a046f8ce7c4 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:32.678646 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c14074-9fcf-4bb7-8cd4-ea9e6fcadb7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:32.679351 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c14074-9fcf-4bb7-8cd4-ea9e6fcadb7f admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:32.679635 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c14074-9fcf-4bb7-8cd4-ea9e6fcadb7f admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:32.679912 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c14074-9fcf-4bb7-8cd4-ea9e6fcadb7f admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:32.680647 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c14074-9fcf-4bb7-8cd4-ea9e6fcadb7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:32.682914 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c14074-9fcf-4bb7-8cd4-ea9e6fcadb7f admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:32.696010 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 994/1975] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:54:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:54:32.700039 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f057bb-6a9b-42a7-a6bc-a544db43268e admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:32.710030 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f057bb-6a9b-42a7-a6bc-a544db43268e admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:32.715864 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d950f994-14c1-4570-8607-e032700443f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:32.716529 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d950f994-14c1-4570-8607-e032700443f0 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:32.716812 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d950f994-14c1-4570-8607-e032700443f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:32.717344 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d950f994-14c1-4570-8607-e032700443f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:32.718064 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d950f994-14c1-4570-8607-e032700443f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:32.719760 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d950f994-14c1-4570-8607-e032700443f0 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:32.735346 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 982/1976] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:54:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:54:32.737484 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c14074-9fcf-4bb7-8cd4-ea9e6fcadb7f admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:32.753373 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c14074-9fcf-4bb7-8cd4-ea9e6fcadb7f admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:32.767287 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d003ace-4468-46d2-b3ae-a56eb5b26936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:32.769133 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d003ace-4468-46d2-b3ae-a56eb5b26936 admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:32.769636 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d003ace-4468-46d2-b3ae-a56eb5b26936 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:32.772632 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d003ace-4468-46d2-b3ae-a56eb5b26936 admin admin] PATH_INFO: `/v3/projects/dc00879def9b424c9b2dac4bc4abe50c/users/1d29c508a87140aaade8a1e4d508bf44/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:32.796675 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d003ace-4468-46d2-b3ae-a56eb5b26936 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc00879def9b424c9b2dac4bc4abe50c, user_id=1d29c508a87140aaade8a1e4d508bf44, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:32.799510 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d003ace-4468-46d2-b3ae-a56eb5b26936 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:32.864875 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 995/1977] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:54:32 2022] PUT /identity/v3/projects/dc00879def9b424c9b2dac4bc4abe50c/users/1d29c508a87140aaade8a1e4d508bf44/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:32.884038 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65685a3c-d3c5-44ab-9325-d57a1414d650 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:32.884326 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65685a3c-d3c5-44ab-9325-d57a1414d650 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:32.884604 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65685a3c-d3c5-44ab-9325-d57a1414d650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:32.924433 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-65685a3c-d3c5-44ab-9325-d57a1414d650 None None] MFA Rules not processed for user `1d29c508a87140aaade8a1e4d508bf44`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:32.925748 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-65685a3c-d3c5-44ab-9325-d57a1414d650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:33.033853 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 983/1978] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:54:33.038192 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d003ace-4468-46d2-b3ae-a56eb5b26936 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:33.123408 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d003ace-4468-46d2-b3ae-a56eb5b26936 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:33.128274 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e9bbc9-61c3-4c9d-9b9f-2cffadb8f486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:33.129002 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e9bbc9-61c3-4c9d-9b9f-2cffadb8f486 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:33.129763 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e9bbc9-61c3-4c9d-9b9f-2cffadb8f486 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:33.130056 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e9bbc9-61c3-4c9d-9b9f-2cffadb8f486 admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:33.130837 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e9bbc9-61c3-4c9d-9b9f-2cffadb8f486 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:33.133803 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e9bbc9-61c3-4c9d-9b9f-2cffadb8f486 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:33.143515 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:33.220528 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 996/1979] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:54:33 2022] POST /identity/v3/projects => generated 373 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:54:33.224722 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65685a3c-d3c5-44ab-9325-d57a1414d650 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:33.235346 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65685a3c-d3c5-44ab-9325-d57a1414d650 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:33.286756 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:33.296702 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e50b003-1d30-4bee-bef5-a0107762ba59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:33.297455 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e50b003-1d30-4bee-bef5-a0107762ba59 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:33.297716 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e50b003-1d30-4bee-bef5-a0107762ba59 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:33.297995 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e50b003-1d30-4bee-bef5-a0107762ba59 admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:33.298729 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e50b003-1d30-4bee-bef5-a0107762ba59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:33.300710 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e50b003-1d30-4bee-bef5-a0107762ba59 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:33.414809 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 984/1980] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:54:33 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:54:33.418336 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e9bbc9-61c3-4c9d-9b9f-2cffadb8f486 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:33.429168 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e9bbc9-61c3-4c9d-9b9f-2cffadb8f486 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:33.434193 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddcf543-392b-44dc-aded-90ff26e9a536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:33.434969 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddcf543-392b-44dc-aded-90ff26e9a536 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:33.435316 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddcf543-392b-44dc-aded-90ff26e9a536 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:33.435654 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddcf543-392b-44dc-aded-90ff26e9a536 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:33.436447 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddcf543-392b-44dc-aded-90ff26e9a536 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:33.438190 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddcf543-392b-44dc-aded-90ff26e9a536 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:33.468429 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 997/1981] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:54:33 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:54:33.475229 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e50b003-1d30-4bee-bef5-a0107762ba59 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:33.486127 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e50b003-1d30-4bee-bef5-a0107762ba59 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:33.502885 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4652c3-7b0c-4e64-8d6b-0b044e86790a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:33.505350 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4652c3-7b0c-4e64-8d6b-0b044e86790a admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:33.505642 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4652c3-7b0c-4e64-8d6b-0b044e86790a admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:33.506003 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4652c3-7b0c-4e64-8d6b-0b044e86790a admin admin] PATH_INFO: `/v3/projects/a3cb5f460cef43679ef6ad5862e4f47e/users/2ca94225ef414e9181ed691aaa5533d2/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:33.528038 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4652c3-7b0c-4e64-8d6b-0b044e86790a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3cb5f460cef43679ef6ad5862e4f47e, user_id=2ca94225ef414e9181ed691aaa5533d2, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:33.531231 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4652c3-7b0c-4e64-8d6b-0b044e86790a admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:33.660198 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 985/1982] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:54:33 2022] PUT /identity/v3/projects/a3cb5f460cef43679ef6ad5862e4f47e/users/2ca94225ef414e9181ed691aaa5533d2/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:33.665101 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddcf543-392b-44dc-aded-90ff26e9a536 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:33.681043 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddcf543-392b-44dc-aded-90ff26e9a536 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:33.755868 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0115ddc1-224f-49aa-ab31-975db45c934d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:33.756714 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115ddc1-224f-49aa-ab31-975db45c934d admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:33.757034 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115ddc1-224f-49aa-ab31-975db45c934d admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:33.757374 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115ddc1-224f-49aa-ab31-975db45c934d admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:33.758209 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0115ddc1-224f-49aa-ab31-975db45c934d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:33.760094 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0115ddc1-224f-49aa-ab31-975db45c934d admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:33.771168 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 998/1983] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:54:33 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:54:33.775278 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4652c3-7b0c-4e64-8d6b-0b044e86790a admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:33.786513 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4652c3-7b0c-4e64-8d6b-0b044e86790a admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:33.791286 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ab727f-e92b-41a9-b281-bc33157fbcc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:33.792288 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ab727f-e92b-41a9-b281-bc33157fbcc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:33.792668 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ab727f-e92b-41a9-b281-bc33157fbcc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:33.792950 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ab727f-e92b-41a9-b281-bc33157fbcc6 admin admin] PATH_INFO: `/v3/projects/a3cb5f460cef43679ef6ad5862e4f47e/users/2ca94225ef414e9181ed691aaa5533d2/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:33.794984 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ab727f-e92b-41a9-b281-bc33157fbcc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3cb5f460cef43679ef6ad5862e4f47e, user_id=2ca94225ef414e9181ed691aaa5533d2, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:33.797612 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ab727f-e92b-41a9-b281-bc33157fbcc6 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:33.808955 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 986/1984] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:54:33 2022] PUT /identity/v3/projects/a3cb5f460cef43679ef6ad5862e4f47e/users/2ca94225ef414e9181ed691aaa5533d2/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:33.819458 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c5eab2-38c2-4a34-918a-773dcc4c1752 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:33.820341 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c5eab2-38c2-4a34-918a-773dcc4c1752 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:33.820341 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c5eab2-38c2-4a34-918a-773dcc4c1752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:33.849710 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-57c5eab2-38c2-4a34-918a-773dcc4c1752 None None] MFA Rules not processed for user `2ca94225ef414e9181ed691aaa5533d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:33.851083 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-57c5eab2-38c2-4a34-918a-773dcc4c1752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:33.928860 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 999/1985] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:33 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:54:33.939550 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ab727f-e92b-41a9-b281-bc33157fbcc6 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:33.941257 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-67ab727f-e92b-41a9-b281-bc33157fbcc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:34.002167 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ab727f-e92b-41a9-b281-bc33157fbcc6 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:34.005928 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2af918-e4fc-419b-b0e8-3a937c25ab7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:34.006575 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2af918-e4fc-419b-b0e8-3a937c25ab7c service cinder] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:34.006893 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2af918-e4fc-419b-b0e8-3a937c25ab7c service cinder] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:34.007171 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2af918-e4fc-419b-b0e8-3a937c25ab7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:34.066689 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2af918-e4fc-419b-b0e8-3a937c25ab7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:34.068150 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2af918-e4fc-419b-b0e8-3a937c25ab7c service cinder] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:34.078311 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 987/1986] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:54:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:54:39.272514 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c5eab2-38c2-4a34-918a-773dcc4c1752 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:39.401701 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c5eab2-38c2-4a34-918a-773dcc4c1752 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:39.406431 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8e2c70-67fa-48e3-9f9e-8bc66b40c273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:39.407362 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e2c70-67fa-48e3-9f9e-8bc66b40c273 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:39.407646 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e2c70-67fa-48e3-9f9e-8bc66b40c273 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:39.407924 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e2c70-67fa-48e3-9f9e-8bc66b40c273 admin admin] PATH_INFO: `/v3/users/1d29c508a87140aaade8a1e4d508bf44` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:39.409152 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8e2c70-67fa-48e3-9f9e-8bc66b40c273 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d29c508a87140aaade8a1e4d508bf44)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:39.411295 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8e2c70-67fa-48e3-9f9e-8bc66b40c273 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:39.413639 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:39.591360 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-1e8e2c70-67fa-48e3-9f9e-8bc66b40c273 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d29c508a87140aaade8a1e4d508bf44'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:39.603221 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-1e8e2c70-67fa-48e3-9f9e-8bc66b40c273 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d29c508a87140aaade8a1e4d508bf44'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:39.603430 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-1e8e2c70-67fa-48e3-9f9e-8bc66b40c273 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d29c508a87140aaade8a1e4d508bf44'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:39.620341 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2af918-e4fc-419b-b0e8-3a937c25ab7c service cinder] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:39.622214 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-1e8e2c70-67fa-48e3-9f9e-8bc66b40c273 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d29c508a87140aaade8a1e4d508bf44'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:39.622536 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-ee2af918-e4fc-419b-b0e8-3a937c25ab7c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:39.637688 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2af918-e4fc-419b-b0e8-3a937c25ab7c service cinder] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:39.642897 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9131b5ba-0f34-429b-8f3d-031acdcc9cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:39.643867 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9131b5ba-0f34-429b-8f3d-031acdcc9cf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:39.644155 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9131b5ba-0f34-429b-8f3d-031acdcc9cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:39.644435 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9131b5ba-0f34-429b-8f3d-031acdcc9cf9 admin admin] PATH_INFO: `/v3/users/b84b170c0bb3438885fe2ff890c5190b` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:39.645774 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9131b5ba-0f34-429b-8f3d-031acdcc9cf9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b84b170c0bb3438885fe2ff890c5190b)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:39.648608 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9131b5ba-0f34-429b-8f3d-031acdcc9cf9 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:39.696595 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-1e8e2c70-67fa-48e3-9f9e-8bc66b40c273 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d29c508a87140aaade8a1e4d508bf44'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:39.703531 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1000/1987] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:54:39 2022] DELETE /identity/v3/users/1d29c508a87140aaade8a1e4d508bf44 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:39.707897 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a4f30-9349-4496-9983-106c8da183b7 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:39.708190 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a4f30-9349-4496-9983-106c8da183b7 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:39.708576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a4f30-9349-4496-9983-106c8da183b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:39.727951 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-273a4f30-9349-4496-9983-106c8da183b7 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:39.729169 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-273a4f30-9349-4496-9983-106c8da183b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:39.797577 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1001/1988] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:54:39.806015 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273a4f30-9349-4496-9983-106c8da183b7 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:39.807441 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-273a4f30-9349-4496-9983-106c8da183b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:39.815423 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-9131b5ba-0f34-429b-8f3d-031acdcc9cf9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b84b170c0bb3438885fe2ff890c5190b'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:39.823041 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:39.825823 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-9131b5ba-0f34-429b-8f3d-031acdcc9cf9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b84b170c0bb3438885fe2ff890c5190b'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:39.826731 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-9131b5ba-0f34-429b-8f3d-031acdcc9cf9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b84b170c0bb3438885fe2ff890c5190b'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:39.842981 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-9131b5ba-0f34-429b-8f3d-031acdcc9cf9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b84b170c0bb3438885fe2ff890c5190b'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:39.844060 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-9131b5ba-0f34-429b-8f3d-031acdcc9cf9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b84b170c0bb3438885fe2ff890c5190b'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:39.879263 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273a4f30-9349-4496-9983-106c8da183b7 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:39.884344 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1122a9e6-cf12-4c96-8677-96fd565fda93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:39.884830 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1122a9e6-cf12-4c96-8677-96fd565fda93 service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:39.885194 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1122a9e6-cf12-4c96-8677-96fd565fda93 service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:39.885609 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1122a9e6-cf12-4c96-8677-96fd565fda93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:39.887670 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-1122a9e6-cf12-4c96-8677-96fd565fda93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:39.902954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 988/1989] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:54:39 2022] DELETE /identity/v3/users/b84b170c0bb3438885fe2ff890c5190b => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:39.911064 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1122a9e6-cf12-4c96-8677-96fd565fda93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:39.911167 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f99ff51-1a36-4e26-b777-17321fe3347d None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:39.911256 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1122a9e6-cf12-4c96-8677-96fd565fda93 service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:39.911359 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f99ff51-1a36-4e26-b777-17321fe3347d None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:39.911981 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f99ff51-1a36-4e26-b777-17321fe3347d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:39.929345 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1002/1990] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:54:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:54:39.932701 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-1f99ff51-1a36-4e26-b777-17321fe3347d None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:39.934031 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-1f99ff51-1a36-4e26-b777-17321fe3347d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:40.020282 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 989/1991] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:54:40.033027 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1122a9e6-cf12-4c96-8677-96fd565fda93 service neutron] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:40.034748 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-1122a9e6-cf12-4c96-8677-96fd565fda93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:40.045911 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1122a9e6-cf12-4c96-8677-96fd565fda93 service neutron] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:40.065220 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:40.095955 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9f32aa-15d8-4f3c-9ceb-6aae48aacadf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:40.096608 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9f32aa-15d8-4f3c-9ceb-6aae48aacadf service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:40.096898 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9f32aa-15d8-4f3c-9ceb-6aae48aacadf service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:40.097178 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9f32aa-15d8-4f3c-9ceb-6aae48aacadf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:40.106449 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9f32aa-15d8-4f3c-9ceb-6aae48aacadf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:40.107917 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9f32aa-15d8-4f3c-9ceb-6aae48aacadf service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:40.118130 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1003/1992] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:54:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:54:40.484922 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f99ff51-1a36-4e26-b777-17321fe3347d None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:40.486619 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-1f99ff51-1a36-4e26-b777-17321fe3347d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:40.543680 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f99ff51-1a36-4e26-b777-17321fe3347d None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:40.568622 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c43ff3-6570-470f-b926-b906f3482b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:40.571561 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c43ff3-6570-470f-b926-b906f3482b04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:40.573177 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c43ff3-6570-470f-b926-b906f3482b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:40.574320 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c43ff3-6570-470f-b926-b906f3482b04 admin admin] PATH_INFO: `/v3/projects/dc00879def9b424c9b2dac4bc4abe50c` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:40.579262 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c43ff3-6570-470f-b926-b906f3482b04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc00879def9b424c9b2dac4bc4abe50c)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:40.582048 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c43ff3-6570-470f-b926-b906f3482b04 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:40.708174 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-b5c43ff3-6570-470f-b926-b906f3482b04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc00879def9b424c9b2dac4bc4abe50c', 'request_id': 'req-b5c43ff3-6570-470f-b926-b906f3482b04'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:40.714760 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-b5c43ff3-6570-470f-b926-b906f3482b04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc00879def9b424c9b2dac4bc4abe50c', 'request_id': 'req-b5c43ff3-6570-470f-b926-b906f3482b04'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:40.784861 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9f32aa-15d8-4f3c-9ceb-6aae48aacadf service neutron] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:40.787397 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-ba9f32aa-15d8-4f3c-9ceb-6aae48aacadf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:40.804284 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-b5c43ff3-6570-470f-b926-b906f3482b04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc00879def9b424c9b2dac4bc4abe50c', 'request_id': 'req-b5c43ff3-6570-470f-b926-b906f3482b04'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:40.804463 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9f32aa-15d8-4f3c-9ceb-6aae48aacadf service neutron] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:40.829624 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-b5c43ff3-6570-470f-b926-b906f3482b04 admin admin] The token cache is being invalidate because project dc00879def9b424c9b2dac4bc4abe50c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:54:40.829624 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-b5c43ff3-6570-470f-b926-b906f3482b04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:40.829624 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-b5c43ff3-6570-470f-b926-b906f3482b04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:40.834826 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 990/1993] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:54:40 2022] DELETE /identity/v3/projects/dc00879def9b424c9b2dac4bc4abe50c => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:40.835878 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c43ff3-6570-470f-b926-b906f3482b04 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:40.837984 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-b5c43ff3-6570-470f-b926-b906f3482b04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:40.883818 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d816a99e-3dd9-40e1-9e0b-499419066a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:40.884623 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d816a99e-3dd9-40e1-9e0b-499419066a82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:40.884912 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d816a99e-3dd9-40e1-9e0b-499419066a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:40.885188 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d816a99e-3dd9-40e1-9e0b-499419066a82 admin admin] PATH_INFO: `/v3/projects/7a8f1e318284416f96e8f609746cde82` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:40.887169 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d816a99e-3dd9-40e1-9e0b-499419066a82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a8f1e318284416f96e8f609746cde82)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:40.889575 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d816a99e-3dd9-40e1-9e0b-499419066a82 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:40.952655 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:40.981387 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c43ff3-6570-470f-b926-b906f3482b04 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:40.987785 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b3808e-1c79-4437-8244-eba31dbf1cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:40.990294 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3808e-1c79-4437-8244-eba31dbf1cd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:40.991487 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3808e-1c79-4437-8244-eba31dbf1cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:40.992697 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3808e-1c79-4437-8244-eba31dbf1cd8 admin admin] PATH_INFO: `/v3/users/2ca94225ef414e9181ed691aaa5533d2` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:40.994601 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b3808e-1c79-4437-8244-eba31dbf1cd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ca94225ef414e9181ed691aaa5533d2)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:40.999990 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b3808e-1c79-4437-8244-eba31dbf1cd8 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:41.058521 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-d816a99e-3dd9-40e1-9e0b-499419066a82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a8f1e318284416f96e8f609746cde82', 'request_id': 'req-d816a99e-3dd9-40e1-9e0b-499419066a82'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:41.065859 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-d816a99e-3dd9-40e1-9e0b-499419066a82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a8f1e318284416f96e8f609746cde82', 'request_id': 'req-d816a99e-3dd9-40e1-9e0b-499419066a82'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:41.159774 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-d816a99e-3dd9-40e1-9e0b-499419066a82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a8f1e318284416f96e8f609746cde82', 'request_id': 'req-d816a99e-3dd9-40e1-9e0b-499419066a82'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:41.171980 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-d816a99e-3dd9-40e1-9e0b-499419066a82 admin admin] The token cache is being invalidate because project 7a8f1e318284416f96e8f609746cde82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:54:41.172261 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-d816a99e-3dd9-40e1-9e0b-499419066a82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:41.172916 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-d816a99e-3dd9-40e1-9e0b-499419066a82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:41.174459 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1004/1994] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:54:40 2022] DELETE /identity/v3/projects/7a8f1e318284416f96e8f609746cde82 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:54:41.178279 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d816a99e-3dd9-40e1-9e0b-499419066a82 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:41.179610 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-d816a99e-3dd9-40e1-9e0b-499419066a82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:41.241174 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:41.246469 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d816a99e-3dd9-40e1-9e0b-499419066a82 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:41.253047 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f1d9b-11ab-4b0b-b097-128ca075ba5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:41.255787 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f1d9b-11ab-4b0b-b097-128ca075ba5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:41.255787 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f1d9b-11ab-4b0b-b097-128ca075ba5b admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:41.255787 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f1d9b-11ab-4b0b-b097-128ca075ba5b admin admin] PATH_INFO: `/v3/users/878e9ac29631475fb30ebac5b172b190` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:41.256778 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078f1d9b-11ab-4b0b-b097-128ca075ba5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=878e9ac29631475fb30ebac5b172b190)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:41.259030 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078f1d9b-11ab-4b0b-b097-128ca075ba5b admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:41.265348 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-70b3808e-1c79-4437-8244-eba31dbf1cd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ca94225ef414e9181ed691aaa5533d2'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:41.274032 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-70b3808e-1c79-4437-8244-eba31dbf1cd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ca94225ef414e9181ed691aaa5533d2'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:41.282270 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-70b3808e-1c79-4437-8244-eba31dbf1cd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ca94225ef414e9181ed691aaa5533d2'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:41.301934 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-70b3808e-1c79-4437-8244-eba31dbf1cd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ca94225ef414e9181ed691aaa5533d2'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:41.302757 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-70b3808e-1c79-4437-8244-eba31dbf1cd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ca94225ef414e9181ed691aaa5533d2'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:41.389418 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 991/1995] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:54:40 2022] DELETE /identity/v3/users/2ca94225ef414e9181ed691aaa5533d2 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:54:41.466507 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-078f1d9b-11ab-4b0b-b097-128ca075ba5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '878e9ac29631475fb30ebac5b172b190'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:41.486922 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-078f1d9b-11ab-4b0b-b097-128ca075ba5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '878e9ac29631475fb30ebac5b172b190'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:41.488041 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-078f1d9b-11ab-4b0b-b097-128ca075ba5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '878e9ac29631475fb30ebac5b172b190'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:41.519832 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-078f1d9b-11ab-4b0b-b097-128ca075ba5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '878e9ac29631475fb30ebac5b172b190'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:41.615355 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-078f1d9b-11ab-4b0b-b097-128ca075ba5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '878e9ac29631475fb30ebac5b172b190'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:41.621958 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1005/1996] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:54:41 2022] DELETE /identity/v3/users/878e9ac29631475fb30ebac5b172b190 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:54:42.012063 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b3808e-1c79-4437-8244-eba31dbf1cd8 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:42.014127 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-70b3808e-1c79-4437-8244-eba31dbf1cd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:42.174543 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b3808e-1c79-4437-8244-eba31dbf1cd8 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:42.250421 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a108cd-d813-44b4-a7b8-28be268affac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:42.251369 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a108cd-d813-44b4-a7b8-28be268affac admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:42.251968 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a108cd-d813-44b4-a7b8-28be268affac admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:42.251968 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a108cd-d813-44b4-a7b8-28be268affac admin admin] PATH_INFO: `/v3/projects/a3cb5f460cef43679ef6ad5862e4f47e` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:42.253387 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a108cd-d813-44b4-a7b8-28be268affac admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3cb5f460cef43679ef6ad5862e4f47e)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:42.255947 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a108cd-d813-44b4-a7b8-28be268affac admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:42.281525 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f1d9b-11ab-4b0b-b097-128ca075ba5b admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:42.283327 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-078f1d9b-11ab-4b0b-b097-128ca075ba5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:42.283748 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:42.317121 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f1d9b-11ab-4b0b-b097-128ca075ba5b admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:42.323156 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c767f844-8e2c-46ff-a729-427e290e9c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:42.324457 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c767f844-8e2c-46ff-a729-427e290e9c8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:42.324973 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c767f844-8e2c-46ff-a729-427e290e9c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:42.325376 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c767f844-8e2c-46ff-a729-427e290e9c8e admin admin] PATH_INFO: `/v3/projects/51e60ebdb650445e9e3b996f846ef474` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:42.326991 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c767f844-8e2c-46ff-a729-427e290e9c8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=51e60ebdb650445e9e3b996f846ef474)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:42.329513 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c767f844-8e2c-46ff-a729-427e290e9c8e admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:42.402655 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-02a108cd-d813-44b4-a7b8-28be268affac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3cb5f460cef43679ef6ad5862e4f47e', 'request_id': 'req-02a108cd-d813-44b4-a7b8-28be268affac'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:42.411388 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-02a108cd-d813-44b4-a7b8-28be268affac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3cb5f460cef43679ef6ad5862e4f47e', 'request_id': 'req-02a108cd-d813-44b4-a7b8-28be268affac'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:42.488104 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-c767f844-8e2c-46ff-a729-427e290e9c8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51e60ebdb650445e9e3b996f846ef474', 'request_id': 'req-c767f844-8e2c-46ff-a729-427e290e9c8e'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:42.504001 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-c767f844-8e2c-46ff-a729-427e290e9c8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51e60ebdb650445e9e3b996f846ef474', 'request_id': 'req-c767f844-8e2c-46ff-a729-427e290e9c8e'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:42.517882 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-02a108cd-d813-44b4-a7b8-28be268affac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3cb5f460cef43679ef6ad5862e4f47e', 'request_id': 'req-02a108cd-d813-44b4-a7b8-28be268affac'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:42.535848 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-02a108cd-d813-44b4-a7b8-28be268affac admin admin] The token cache is being invalidate because project a3cb5f460cef43679ef6ad5862e4f47e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:54:42.536226 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-02a108cd-d813-44b4-a7b8-28be268affac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:42.536926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-02a108cd-d813-44b4-a7b8-28be268affac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:42.538515 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 992/1997] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:54:42 2022] DELETE /identity/v3/projects/a3cb5f460cef43679ef6ad5862e4f47e => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:42.543764 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a9c16-b7f5-4a94-b557-15cb7dcd56a5 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:42.544059 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a9c16-b7f5-4a94-b557-15cb7dcd56a5 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:42.544339 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a9c16-b7f5-4a94-b557-15cb7dcd56a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:42.563627 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-213a9c16-b7f5-4a94-b557-15cb7dcd56a5 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:42.564848 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-213a9c16-b7f5-4a94-b557-15cb7dcd56a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:42.605074 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-c767f844-8e2c-46ff-a729-427e290e9c8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51e60ebdb650445e9e3b996f846ef474', 'request_id': 'req-c767f844-8e2c-46ff-a729-427e290e9c8e'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:42.620515 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-c767f844-8e2c-46ff-a729-427e290e9c8e admin admin] The token cache is being invalidate because project 51e60ebdb650445e9e3b996f846ef474 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:54:42.620920 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-c767f844-8e2c-46ff-a729-427e290e9c8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:42.621663 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-c767f844-8e2c-46ff-a729-427e290e9c8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:42.623342 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1006/1998] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:54:42 2022] DELETE /identity/v3/projects/51e60ebdb650445e9e3b996f846ef474 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:42.629305 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2a56c-8731-46b5-a494-bbce45ef35b2 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:42.629305 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2a56c-8731-46b5-a494-bbce45ef35b2 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:42.629305 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2a56c-8731-46b5-a494-bbce45ef35b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:42.639381 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 993/1999] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:54:42.648983 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f1b0ed-54d5-4ea1-ba93-8c575e9ae560 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:42.649488 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f1b0ed-54d5-4ea1-ba93-8c575e9ae560 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:42.649764 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f1b0ed-54d5-4ea1-ba93-8c575e9ae560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:42.655659 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-83a2a56c-8731-46b5-a494-bbce45ef35b2 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:42.656892 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-83a2a56c-8731-46b5-a494-bbce45ef35b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:42.666923 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1007/2000] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:54:42.671580 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-49f1b0ed-54d5-4ea1-ba93-8c575e9ae560 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:42.673208 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-49f1b0ed-54d5-4ea1-ba93-8c575e9ae560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:42.674496 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee2480-0b8b-42f5-bb69-766a3dd06499 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:42.675446 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee2480-0b8b-42f5-bb69-766a3dd06499 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:42.676283 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee2480-0b8b-42f5-bb69-766a3dd06499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:42.682044 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 994/2001] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:54:42.691529 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f1b0ed-54d5-4ea1-ba93-8c575e9ae560 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:42.706858 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f1b0ed-54d5-4ea1-ba93-8c575e9ae560 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:42.716237 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2d3ed3-a2c9-435a-9c38-05f0f6e886a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:42.720749 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2d3ed3-a2c9-435a-9c38-05f0f6e886a7 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:42.720749 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2d3ed3-a2c9-435a-9c38-05f0f6e886a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:42.720749 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2d3ed3-a2c9-435a-9c38-05f0f6e886a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:42.720749 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2d3ed3-a2c9-435a-9c38-05f0f6e886a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:54:42.720749 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2d3ed3-a2c9-435a-9c38-05f0f6e886a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:42.721137 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-77ee2480-0b8b-42f5-bb69-766a3dd06499 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:42.721241 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-77ee2480-0b8b-42f5-bb69-766a3dd06499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:42.721530 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2d3ed3-a2c9-435a-9c38-05f0f6e886a7 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:42.731944 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1008/2002] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:54:42.736131 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ee2480-0b8b-42f5-bb69-766a3dd06499 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:42.738712 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 995/2003] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 22:54:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:54:42.746273 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ee2480-0b8b-42f5-bb69-766a3dd06499 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:42.748970 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2d3ed3-a2c9-435a-9c38-05f0f6e886a7 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:42.753241 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b53130-ef74-4ac9-8a32-9c97df4ef4d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:42.754000 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b53130-ef74-4ac9-8a32-9c97df4ef4d0 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:42.754293 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b53130-ef74-4ac9-8a32-9c97df4ef4d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:42.754647 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b53130-ef74-4ac9-8a32-9c97df4ef4d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:42.755598 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b53130-ef74-4ac9-8a32-9c97df4ef4d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:54:42.755957 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b53130-ef74-4ac9-8a32-9c97df4ef4d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:42.757782 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b53130-ef74-4ac9-8a32-9c97df4ef4d0 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:42.762490 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2d3ed3-a2c9-435a-9c38-05f0f6e886a7 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:42.768927 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d9b533-9ba6-4094-821c-5bc5710faaa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:42.770308 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9b533-9ba6-4094-821c-5bc5710faaa6 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:42.774404 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9b533-9ba6-4094-821c-5bc5710faaa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:42.774404 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9b533-9ba6-4094-821c-5bc5710faaa6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:42.774404 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d9b533-9ba6-4094-821c-5bc5710faaa6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:42.775974 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d9b533-9ba6-4094-821c-5bc5710faaa6 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:42.778666 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1009/2004] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 22:54:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:54:42.782896 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b53130-ef74-4ac9-8a32-9c97df4ef4d0 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:42.794628 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b53130-ef74-4ac9-8a32-9c97df4ef4d0 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:42.801606 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed010d0-9d72-4949-bcbb-834c7d3883e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:42.807318 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed010d0-9d72-4949-bcbb-834c7d3883e1 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:42.807318 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed010d0-9d72-4949-bcbb-834c7d3883e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:42.807318 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed010d0-9d72-4949-bcbb-834c7d3883e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:42.807318 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed010d0-9d72-4949-bcbb-834c7d3883e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:42.807318 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed010d0-9d72-4949-bcbb-834c7d3883e1 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:42.869591 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 996/2005] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:54:42 2022] POST /identity/v3/projects => generated 385 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:54:42.875885 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d9b533-9ba6-4094-821c-5bc5710faaa6 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:42.920450 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1010/2006] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:54:42 2022] POST /identity/v3/projects => generated 365 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:54:42.921617 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed010d0-9d72-4949-bcbb-834c7d3883e1 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:42.945959 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:42.972319 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d9b533-9ba6-4094-821c-5bc5710faaa6 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:42.979963 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:43.044620 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed010d0-9d72-4949-bcbb-834c7d3883e1 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:43.052963 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d307ecf3-cd1a-4988-bdf9-ad83f75baa5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:43.053656 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307ecf3-cd1a-4988-bdf9-ad83f75baa5e admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:43.053949 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307ecf3-cd1a-4988-bdf9-ad83f75baa5e admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:43.054279 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307ecf3-cd1a-4988-bdf9-ad83f75baa5e admin admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:43.055142 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d307ecf3-cd1a-4988-bdf9-ad83f75baa5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:43.058251 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d307ecf3-cd1a-4988-bdf9-ad83f75baa5e admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:43.071103 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df486ca-8da4-4f46-bf5a-42ba043faca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:43.073115 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df486ca-8da4-4f46-bf5a-42ba043faca7 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:43.073562 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df486ca-8da4-4f46-bf5a-42ba043faca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:43.074039 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df486ca-8da4-4f46-bf5a-42ba043faca7 admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:43.074747 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df486ca-8da4-4f46-bf5a-42ba043faca7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:43.076831 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df486ca-8da4-4f46-bf5a-42ba043faca7 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:43.153634 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1011/2007] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:54:42 2022] POST /identity/v3/users => generated 283 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:54:43.154952 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d307ecf3-cd1a-4988-bdf9-ad83f75baa5e admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:43.167230 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d307ecf3-cd1a-4988-bdf9-ad83f75baa5e admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:43.173120 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a876c3cf-f08a-45d8-a02e-ae979549bc8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:43.173953 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a876c3cf-f08a-45d8-a02e-ae979549bc8f admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:43.174241 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a876c3cf-f08a-45d8-a02e-ae979549bc8f admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:43.174523 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a876c3cf-f08a-45d8-a02e-ae979549bc8f admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:43.177002 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a876c3cf-f08a-45d8-a02e-ae979549bc8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:43.179017 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a876c3cf-f08a-45d8-a02e-ae979549bc8f admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:43.183278 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 997/2008] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:54:42 2022] POST /identity/v3/users => generated 293 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:54:43.192866 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1012/2009] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:54:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:54:43.193042 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df486ca-8da4-4f46-bf5a-42ba043faca7 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:43.193840 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a876c3cf-f08a-45d8-a02e-ae979549bc8f admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:43.203118 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df486ca-8da4-4f46-bf5a-42ba043faca7 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:43.204134 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a876c3cf-f08a-45d8-a02e-ae979549bc8f admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:43.209144 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb53176-21fb-4481-831f-dd6e7b077ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:43.209514 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66178da7-ecc4-40e6-8d9a-5c09ae2042b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:43.210175 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66178da7-ecc4-40e6-8d9a-5c09ae2042b1 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:43.210354 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb53176-21fb-4481-831f-dd6e7b077ff1 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:43.210575 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66178da7-ecc4-40e6-8d9a-5c09ae2042b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:43.211000 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66178da7-ecc4-40e6-8d9a-5c09ae2042b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:43.216467 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb53176-21fb-4481-831f-dd6e7b077ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:43.216467 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb53176-21fb-4481-831f-dd6e7b077ff1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:43.217025 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66178da7-ecc4-40e6-8d9a-5c09ae2042b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:43.217124 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb53176-21fb-4481-831f-dd6e7b077ff1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:43.217209 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66178da7-ecc4-40e6-8d9a-5c09ae2042b1 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:43.217309 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb53176-21fb-4481-831f-dd6e7b077ff1 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:43.225295 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1013/2010] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:54:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:54:43.225998 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 998/2011] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:54:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:54:43.229781 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66178da7-ecc4-40e6-8d9a-5c09ae2042b1 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:43.232248 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb53176-21fb-4481-831f-dd6e7b077ff1 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:43.240385 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66178da7-ecc4-40e6-8d9a-5c09ae2042b1 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:43.245439 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472d5cae-a211-4014-a06c-73d148ea1509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:43.246136 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d5cae-a211-4014-a06c-73d148ea1509 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:43.246415 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d5cae-a211-4014-a06c-73d148ea1509 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:43.246884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d5cae-a211-4014-a06c-73d148ea1509 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:43.247529 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472d5cae-a211-4014-a06c-73d148ea1509 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:43.249151 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472d5cae-a211-4014-a06c-73d148ea1509 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:43.251372 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb53176-21fb-4481-831f-dd6e7b077ff1 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:43.257368 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a533359-68ed-453a-876d-f1d624e22c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:43.258468 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a533359-68ed-453a-876d-f1d624e22c27 admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:43.259111 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a533359-68ed-453a-876d-f1d624e22c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:43.259520 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a533359-68ed-453a-876d-f1d624e22c27 admin admin] PATH_INFO: `/v3/projects/f3747675d8d34576ad27b1eff96aabcc/users/c4b3ac2700db452f9973978b8b97ac70/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:43.266811 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1014/2012] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:54:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:54:43.267623 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472d5cae-a211-4014-a06c-73d148ea1509 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:43.287057 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a533359-68ed-453a-876d-f1d624e22c27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3747675d8d34576ad27b1eff96aabcc, user_id=c4b3ac2700db452f9973978b8b97ac70, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:43.290383 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a533359-68ed-453a-876d-f1d624e22c27 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:43.301570 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472d5cae-a211-4014-a06c-73d148ea1509 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:43.308005 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e112c3-d0d8-497c-b4a1-dfa497ec58e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:43.308598 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e112c3-d0d8-497c-b4a1-dfa497ec58e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:43.308868 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e112c3-d0d8-497c-b4a1-dfa497ec58e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:43.309186 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e112c3-d0d8-497c-b4a1-dfa497ec58e1 admin admin] PATH_INFO: `/v3/projects/1adf6d8fe0774070a02b77d1e712990b/users/137935860b364d698866aa5388668db8/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:43.330742 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e112c3-d0d8-497c-b4a1-dfa497ec58e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1adf6d8fe0774070a02b77d1e712990b, user_id=137935860b364d698866aa5388668db8, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:43.335116 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e112c3-d0d8-497c-b4a1-dfa497ec58e1 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:43.396913 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 999/2013] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:54:43 2022] PUT /identity/v3/projects/f3747675d8d34576ad27b1eff96aabcc/users/c4b3ac2700db452f9973978b8b97ac70/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:54:43.418965 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4bae99-aa39-4463-a5fe-3caa9eb2340b None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:43.419631 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4bae99-aa39-4463-a5fe-3caa9eb2340b None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:43.419631 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4bae99-aa39-4463-a5fe-3caa9eb2340b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:43.428519 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1015/2014] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:54:43 2022] PUT /identity/v3/projects/1adf6d8fe0774070a02b77d1e712990b/users/137935860b364d698866aa5388668db8/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:43.442332 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732f35d-345e-427f-be7b-94c338388a39 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:43.442332 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732f35d-345e-427f-be7b-94c338388a39 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:43.444297 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732f35d-345e-427f-be7b-94c338388a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:43.482326 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-8a4bae99-aa39-4463-a5fe-3caa9eb2340b None None] MFA Rules not processed for user `c4b3ac2700db452f9973978b8b97ac70`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:43.490195 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-b732f35d-345e-427f-be7b-94c338388a39 None None] MFA Rules not processed for user `137935860b364d698866aa5388668db8`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:43.490899 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-8a4bae99-aa39-4463-a5fe-3caa9eb2340b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:43.492642 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-b732f35d-345e-427f-be7b-94c338388a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:43.583726 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1016/2015] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:54:43.584885 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b732f35d-345e-427f-be7b-94c338388a39 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:43.615661 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1000/2016] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:43 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:54:43.619425 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4bae99-aa39-4463-a5fe-3caa9eb2340b None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:43.644833 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:43.675804 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b732f35d-345e-427f-be7b-94c338388a39 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:43.680573 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8ddda5-fe49-44f5-b8bf-63ca501c6350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:43.681313 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8ddda5-fe49-44f5-b8bf-63ca501c6350 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:43.681600 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8ddda5-fe49-44f5-b8bf-63ca501c6350 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:43.681877 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8ddda5-fe49-44f5-b8bf-63ca501c6350 admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:43.682629 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8ddda5-fe49-44f5-b8bf-63ca501c6350 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:43.685489 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8ddda5-fe49-44f5-b8bf-63ca501c6350 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:43.693468 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4bae99-aa39-4463-a5fe-3caa9eb2340b None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:43.698019 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb674ee-9343-4850-9331-23c37b0a24a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:43.699068 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb674ee-9343-4850-9331-23c37b0a24a9 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:43.699349 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb674ee-9343-4850-9331-23c37b0a24a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:43.699622 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb674ee-9343-4850-9331-23c37b0a24a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:43.700310 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb674ee-9343-4850-9331-23c37b0a24a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:43.702362 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb674ee-9343-4850-9331-23c37b0a24a9 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:43.705825 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:43.820059 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1001/2017] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:54:43 2022] POST /identity/v3/projects => generated 365 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:54:43.821388 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1017/2018] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:54:43 2022] POST /identity/v3/projects => generated 383 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:54:43.822293 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b8b95-f46d-413d-aa47-4d1308ee9c4d None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:43.823906 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b8b95-f46d-413d-aa47-4d1308ee9c4d None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:43.823906 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b8b95-f46d-413d-aa47-4d1308ee9c4d None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:43.824795 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1002/2019] 10.222.0.52 () {54 vars in 921 bytes} [Thu Jul 21 22:54:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:54:43.826052 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680b8b95-f46d-413d-aa47-4d1308ee9c4d None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:43.826379 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8ddda5-fe49-44f5-b8bf-63ca501c6350 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:43.912700 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8ddda5-fe49-44f5-b8bf-63ca501c6350 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:43.919233 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680b8b95-f46d-413d-aa47-4d1308ee9c4d None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:43.919398 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3840a166-b1e5-4402-add2-7e58bbaba55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:43.919495 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a59030-599f-4c97-93ed-c0d3181ebe2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:43.919620 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3840a166-b1e5-4402-add2-7e58bbaba55f admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:43.919695 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a59030-599f-4c97-93ed-c0d3181ebe2d admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:43.919764 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3840a166-b1e5-4402-add2-7e58bbaba55f admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:43.919831 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a59030-599f-4c97-93ed-c0d3181ebe2d admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:43.919912 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3840a166-b1e5-4402-add2-7e58bbaba55f admin admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:43.919991 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a59030-599f-4c97-93ed-c0d3181ebe2d admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:43.920061 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3840a166-b1e5-4402-add2-7e58bbaba55f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:43.920130 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a59030-599f-4c97-93ed-c0d3181ebe2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:43.921534 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3840a166-b1e5-4402-add2-7e58bbaba55f admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:43.921790 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a59030-599f-4c97-93ed-c0d3181ebe2d admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:44.212680 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1003/2020] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:54:43 2022] POST /identity/v3/users => generated 292 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:54:44.215930 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d37b8-cbcb-440b-9acd-345c9726b224 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:44.216561 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d37b8-cbcb-440b-9acd-345c9726b224 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:44.217201 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d37b8-cbcb-440b-9acd-345c9726b224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:44.217547 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1018/2021] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:54:43 2022] POST /identity/v3/users => generated 289 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:54:44.221346 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3840a166-b1e5-4402-add2-7e58bbaba55f admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:44.240842 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3840a166-b1e5-4402-add2-7e58bbaba55f admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:44.249353 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e63715-1f3e-41e2-8880-3c870170a384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:44.249353 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e63715-1f3e-41e2-8880-3c870170a384 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:44.249353 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e63715-1f3e-41e2-8880-3c870170a384 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:44.249353 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e63715-1f3e-41e2-8880-3c870170a384 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:44.249687 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-b49d37b8-cbcb-440b-9acd-345c9726b224 None None] MFA Rules not processed for user `10c0bed169344a77bfad29624c5e5a8e`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:44.249787 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e63715-1f3e-41e2-8880-3c870170a384 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:44.249875 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-b49d37b8-cbcb-440b-9acd-345c9726b224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:44.250368 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e63715-1f3e-41e2-8880-3c870170a384 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:44.264519 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1019/2022] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:54:44 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:54:44.264519 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e63715-1f3e-41e2-8880-3c870170a384 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:44.272269 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e63715-1f3e-41e2-8880-3c870170a384 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:44.276837 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf8f1b9-81ce-4cf6-8458-bece16ac2ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:44.277489 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8f1b9-81ce-4cf6-8458-bece16ac2ffa admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:44.277783 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8f1b9-81ce-4cf6-8458-bece16ac2ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:44.278308 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8f1b9-81ce-4cf6-8458-bece16ac2ffa admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:44.278849 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf8f1b9-81ce-4cf6-8458-bece16ac2ffa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:44.283741 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf8f1b9-81ce-4cf6-8458-bece16ac2ffa admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:44.291652 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1020/2023] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:54:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:54:44.292469 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf8f1b9-81ce-4cf6-8458-bece16ac2ffa admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:44.294155 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-6bf8f1b9-81ce-4cf6-8458-bece16ac2ffa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:44.307317 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf8f1b9-81ce-4cf6-8458-bece16ac2ffa admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:44.311963 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87147537-3a52-4b60-b863-2d68d5375e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:44.312851 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87147537-3a52-4b60-b863-2d68d5375e38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:44.313148 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87147537-3a52-4b60-b863-2d68d5375e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:44.313406 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87147537-3a52-4b60-b863-2d68d5375e38 admin admin] PATH_INFO: `/v3/users/ca0902f6348840729a6ce7a97167da9c` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:44.314618 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87147537-3a52-4b60-b863-2d68d5375e38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca0902f6348840729a6ce7a97167da9c)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:44.316533 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87147537-3a52-4b60-b863-2d68d5375e38 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:44.318529 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1004/2024] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Jul 21 22:54:44 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:54:44.327059 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d37b8-cbcb-440b-9acd-345c9726b224 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:44.341295 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d37b8-cbcb-440b-9acd-345c9726b224 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:44.346110 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba4549c-396f-4341-8fdb-8ff17a956729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:44.346802 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4549c-396f-4341-8fdb-8ff17a956729 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:44.347093 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4549c-396f-4341-8fdb-8ff17a956729 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:44.347368 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4549c-396f-4341-8fdb-8ff17a956729 admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:44.348087 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba4549c-396f-4341-8fdb-8ff17a956729 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:44.350270 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba4549c-396f-4341-8fdb-8ff17a956729 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:44.360378 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1005/2025] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:54:44 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:54:44.361156 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba4549c-396f-4341-8fdb-8ff17a956729 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:44.371091 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba4549c-396f-4341-8fdb-8ff17a956729 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:44.375726 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8999879-a5cf-490e-b37c-2e0b6deeb39a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:44.376612 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8999879-a5cf-490e-b37c-2e0b6deeb39a admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:44.376899 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8999879-a5cf-490e-b37c-2e0b6deeb39a admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:44.377180 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8999879-a5cf-490e-b37c-2e0b6deeb39a admin admin] PATH_INFO: `/v3/projects/181f59b2ba7c4854b77624390f9ad1c4/users/89082c9061ef4fabba5869845d348488/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:44.485979 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8999879-a5cf-490e-b37c-2e0b6deeb39a admin admin] RBAC: Authorizing `identity:create_grant(project_id=181f59b2ba7c4854b77624390f9ad1c4, user_id=89082c9061ef4fabba5869845d348488, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:44.488715 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8999879-a5cf-490e-b37c-2e0b6deeb39a admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:44.586709 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1006/2026] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:54:44 2022] PUT /identity/v3/projects/181f59b2ba7c4854b77624390f9ad1c4/users/89082c9061ef4fabba5869845d348488/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:44.586709 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8999879-a5cf-490e-b37c-2e0b6deeb39a admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:44.588707 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-b8999879-a5cf-490e-b37c-2e0b6deeb39a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:44.647906 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-87147537-3a52-4b60-b863-2d68d5375e38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca0902f6348840729a6ce7a97167da9c'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:44.656201 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-87147537-3a52-4b60-b863-2d68d5375e38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca0902f6348840729a6ce7a97167da9c'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:44.658317 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-87147537-3a52-4b60-b863-2d68d5375e38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca0902f6348840729a6ce7a97167da9c'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:44.664100 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8999879-a5cf-490e-b37c-2e0b6deeb39a admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:44.672325 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ac464-9e9c-4971-8152-75154c03ac8c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10c0bed169344a77bfad29624c5e5a8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:44.672325 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ac464-9e9c-4971-8152-75154c03ac8c service placement] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:44.672325 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ac464-9e9c-4971-8152-75154c03ac8c service placement] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:44.672325 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ac464-9e9c-4971-8152-75154c03ac8c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:44.673993 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-87147537-3a52-4b60-b863-2d68d5375e38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca0902f6348840729a6ce7a97167da9c'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:44.680787 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206ac464-9e9c-4971-8152-75154c03ac8c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:44.682421 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206ac464-9e9c-4971-8152-75154c03ac8c service placement] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:44.692700 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1007/2027] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:54:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:54:44.693959 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ac464-9e9c-4971-8152-75154c03ac8c service placement] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:44.720771 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:44.748313 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-87147537-3a52-4b60-b863-2d68d5375e38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca0902f6348840729a6ce7a97167da9c'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:44.755666 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1021/2028] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:54:44 2022] DELETE /identity/v3/users/ca0902f6348840729a6ce7a97167da9c => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:44.758909 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87147537-3a52-4b60-b863-2d68d5375e38 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:44.758909 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-87147537-3a52-4b60-b863-2d68d5375e38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:44.763867 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ac464-9e9c-4971-8152-75154c03ac8c service placement] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:44.830277 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:44.838173 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15564088-c6a5-4e6d-af55-860b9f525628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:44.839207 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15564088-c6a5-4e6d-af55-860b9f525628 admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:44.839491 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15564088-c6a5-4e6d-af55-860b9f525628 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:44.839774 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15564088-c6a5-4e6d-af55-860b9f525628 admin admin] PATH_INFO: `/v3/projects/08ca1a8b470d48d596408d9edd4c69b9/users/8bcc1e865e6549d4a545b5f78791ca2d/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:44.850406 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87147537-3a52-4b60-b863-2d68d5375e38 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:44.856297 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e4d72e-a204-45d4-a785-699af21df483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:44.857416 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15564088-c6a5-4e6d-af55-860b9f525628 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08ca1a8b470d48d596408d9edd4c69b9, user_id=8bcc1e865e6549d4a545b5f78791ca2d, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:44.857540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4d72e-a204-45d4-a785-699af21df483 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:44.857939 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4d72e-a204-45d4-a785-699af21df483 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:44.858309 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4d72e-a204-45d4-a785-699af21df483 admin admin] PATH_INFO: `/v3/users/306adac7c3c3406fb5ab95d6d76eea9a` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:44.859908 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e4d72e-a204-45d4-a785-699af21df483 admin admin] RBAC: Authorizing `identity:delete_user(user_id=306adac7c3c3406fb5ab95d6d76eea9a)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:44.860212 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15564088-c6a5-4e6d-af55-860b9f525628 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:44.862231 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e4d72e-a204-45d4-a785-699af21df483 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:44.960838 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1008/2029] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:54:44 2022] PUT /identity/v3/projects/08ca1a8b470d48d596408d9edd4c69b9/users/8bcc1e865e6549d4a545b5f78791ca2d/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:44.965843 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15564088-c6a5-4e6d-af55-860b9f525628 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:44.972355 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-15564088-c6a5-4e6d-af55-860b9f525628 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:45.036103 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15564088-c6a5-4e6d-af55-860b9f525628 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:45.040817 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56751724-1182-4942-9452-f51d6d2a5720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:45.041486 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56751724-1182-4942-9452-f51d6d2a5720 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:45.041772 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56751724-1182-4942-9452-f51d6d2a5720 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:45.042050 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56751724-1182-4942-9452-f51d6d2a5720 admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:45.042826 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56751724-1182-4942-9452-f51d6d2a5720 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:45.044464 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56751724-1182-4942-9452-f51d6d2a5720 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:45.056261 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1009/2030] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:54:44 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:54:45.056261 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628f0299-3d90-4c87-bc6d-9375a14d48a7 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:45.056261 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628f0299-3d90-4c87-bc6d-9375a14d48a7 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:45.056261 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628f0299-3d90-4c87-bc6d-9375a14d48a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:45.085280 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-628f0299-3d90-4c87-bc6d-9375a14d48a7 None None] MFA Rules not processed for user `8bcc1e865e6549d4a545b5f78791ca2d`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:45.086514 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-628f0299-3d90-4c87-bc6d-9375a14d48a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:45.144881 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-06e4d72e-a204-45d4-a785-699af21df483 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '306adac7c3c3406fb5ab95d6d76eea9a'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:45.150365 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1010/2031] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:54:45.151326 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628f0299-3d90-4c87-bc6d-9375a14d48a7 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:45.152093 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-06e4d72e-a204-45d4-a785-699af21df483 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '306adac7c3c3406fb5ab95d6d76eea9a'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:45.152888 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-06e4d72e-a204-45d4-a785-699af21df483 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '306adac7c3c3406fb5ab95d6d76eea9a'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:45.153293 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-628f0299-3d90-4c87-bc6d-9375a14d48a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:45.166319 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-06e4d72e-a204-45d4-a785-699af21df483 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '306adac7c3c3406fb5ab95d6d76eea9a'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:45.166909 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628f0299-3d90-4c87-bc6d-9375a14d48a7 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:45.171516 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1901f81c-cdee-41a1-bc52-41641f11f1a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:45.172362 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901f81c-cdee-41a1-bc52-41641f11f1a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:45.172669 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901f81c-cdee-41a1-bc52-41641f11f1a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:45.172943 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901f81c-cdee-41a1-bc52-41641f11f1a8 admin admin] PATH_INFO: `/v3/projects/ba0db0569e094eccb3e8fc578a16fc1e` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:45.174111 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1901f81c-cdee-41a1-bc52-41641f11f1a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba0db0569e094eccb3e8fc578a16fc1e)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:45.176024 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1901f81c-cdee-41a1-bc52-41641f11f1a8 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:45.285881 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-06e4d72e-a204-45d4-a785-699af21df483 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '306adac7c3c3406fb5ab95d6d76eea9a'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:45.292516 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1022/2032] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:54:44 2022] DELETE /identity/v3/users/306adac7c3c3406fb5ab95d6d76eea9a => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:45.293279 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e4d72e-a204-45d4-a785-699af21df483 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:45.294732 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-06e4d72e-a204-45d4-a785-699af21df483 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:45.305353 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-1901f81c-cdee-41a1-bc52-41641f11f1a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba0db0569e094eccb3e8fc578a16fc1e', 'request_id': 'req-1901f81c-cdee-41a1-bc52-41641f11f1a8'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:45.310918 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-1901f81c-cdee-41a1-bc52-41641f11f1a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba0db0569e094eccb3e8fc578a16fc1e', 'request_id': 'req-1901f81c-cdee-41a1-bc52-41641f11f1a8'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:45.357953 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e4d72e-a204-45d4-a785-699af21df483 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:45.362623 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5aa158-bd65-4ae6-b2e3-1f7f58bd1e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:45.363541 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5aa158-bd65-4ae6-b2e3-1f7f58bd1e5f admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:45.363824 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5aa158-bd65-4ae6-b2e3-1f7f58bd1e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:45.364097 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5aa158-bd65-4ae6-b2e3-1f7f58bd1e5f admin admin] PATH_INFO: `/v3/projects/181f59b2ba7c4854b77624390f9ad1c4/users/89082c9061ef4fabba5869845d348488/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:45.366586 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5aa158-bd65-4ae6-b2e3-1f7f58bd1e5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=181f59b2ba7c4854b77624390f9ad1c4, user_id=89082c9061ef4fabba5869845d348488, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:45.369246 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5aa158-bd65-4ae6-b2e3-1f7f58bd1e5f admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:45.379212 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1023/2033] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:54:45 2022] PUT /identity/v3/projects/181f59b2ba7c4854b77624390f9ad1c4/users/89082c9061ef4fabba5869845d348488/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:45.381293 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5aa158-bd65-4ae6-b2e3-1f7f58bd1e5f admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:45.382926 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-ec5aa158-bd65-4ae6-b2e3-1f7f58bd1e5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:45.420676 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-1901f81c-cdee-41a1-bc52-41641f11f1a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba0db0569e094eccb3e8fc578a16fc1e', 'request_id': 'req-1901f81c-cdee-41a1-bc52-41641f11f1a8'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:45.436823 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-1901f81c-cdee-41a1-bc52-41641f11f1a8 admin admin] The token cache is being invalidate because project ba0db0569e094eccb3e8fc578a16fc1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:54:45.437113 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-1901f81c-cdee-41a1-bc52-41641f11f1a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:45.437801 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-1901f81c-cdee-41a1-bc52-41641f11f1a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:45.439617 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1011/2034] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:54:45 2022] DELETE /identity/v3/projects/ba0db0569e094eccb3e8fc578a16fc1e => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:45.440723 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59322c79-f4aa-476d-9ae3-b5b5f0599114 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:45.441015 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59322c79-f4aa-476d-9ae3-b5b5f0599114 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:45.441342 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59322c79-f4aa-476d-9ae3-b5b5f0599114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:45.448732 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5aa158-bd65-4ae6-b2e3-1f7f58bd1e5f admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:45.453563 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6791cae3-470a-4d13-91a0-ef05c8b454a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:45.454263 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791cae3-470a-4d13-91a0-ef05c8b454a0 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:45.454545 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791cae3-470a-4d13-91a0-ef05c8b454a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:45.454848 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791cae3-470a-4d13-91a0-ef05c8b454a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:45.455566 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6791cae3-470a-4d13-91a0-ef05c8b454a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:45.457441 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6791cae3-470a-4d13-91a0-ef05c8b454a0 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:45.459722 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-59322c79-f4aa-476d-9ae3-b5b5f0599114 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:45.462763 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-59322c79-f4aa-476d-9ae3-b5b5f0599114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:45.469416 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1012/2035] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:54:45.470732 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104a909-b025-4ca9-89c8-e4e7569ece83 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:45.471050 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104a909-b025-4ca9-89c8-e4e7569ece83 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:45.471325 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104a909-b025-4ca9-89c8-e4e7569ece83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:45.501015 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-a104a909-b025-4ca9-89c8-e4e7569ece83 None None] MFA Rules not processed for user `89082c9061ef4fabba5869845d348488`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:45.502224 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-a104a909-b025-4ca9-89c8-e4e7569ece83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:45.568197 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1013/2036] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:45 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:54:45.569550 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe16717-02a8-4a65-87b6-862261205ce2 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:45.569842 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe16717-02a8-4a65-87b6-862261205ce2 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:45.570148 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe16717-02a8-4a65-87b6-862261205ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:45.577862 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1024/2037] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:54:45 2022] POST /identity/v3/projects => generated 383 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:54:45.578108 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6791cae3-470a-4d13-91a0-ef05c8b454a0 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:45.579840 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-6791cae3-470a-4d13-91a0-ef05c8b454a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:45.590760 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-abe16717-02a8-4a65-87b6-862261205ce2 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:45.592218 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-abe16717-02a8-4a65-87b6-862261205ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:45.637270 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6791cae3-470a-4d13-91a0-ef05c8b454a0 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:45.641105 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f49244-82e9-4c68-9aa4-14c2a9b2bafa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:45.642938 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f49244-82e9-4c68-9aa4-14c2a9b2bafa service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:45.642938 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f49244-82e9-4c68-9aa4-14c2a9b2bafa service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:45.642938 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f49244-82e9-4c68-9aa4-14c2a9b2bafa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:45.651631 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1014/2038] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:54:45.652468 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe16717-02a8-4a65-87b6-862261205ce2 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:45.653717 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-abe16717-02a8-4a65-87b6-862261205ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:45.710174 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe16717-02a8-4a65-87b6-862261205ce2 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:45.711197 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f49244-82e9-4c68-9aa4-14c2a9b2bafa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:45.712683 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f49244-82e9-4c68-9aa4-14c2a9b2bafa service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:45.714395 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a8374-18d6-4a14-a2b6-cd38da77f934 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:45.715081 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a8374-18d6-4a14-a2b6-cd38da77f934 service cinder] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:45.715364 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a8374-18d6-4a14-a2b6-cd38da77f934 service cinder] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:45.715640 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a8374-18d6-4a14-a2b6-cd38da77f934 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:45.722645 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1025/2039] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:54:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:54:45.723457 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f49244-82e9-4c68-9aa4-14c2a9b2bafa service neutron] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:45.725286 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-74f49244-82e9-4c68-9aa4-14c2a9b2bafa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:45.738446 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f49244-82e9-4c68-9aa4-14c2a9b2bafa service neutron] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:45.746911 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a437ba-57e7-423d-ab91-f84953075edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:45.746911 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a437ba-57e7-423d-ab91-f84953075edd admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:45.746911 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a437ba-57e7-423d-ab91-f84953075edd admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:45.746911 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a437ba-57e7-423d-ab91-f84953075edd admin admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:45.747308 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a437ba-57e7-423d-ab91-f84953075edd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:45.747813 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a437ba-57e7-423d-ab91-f84953075edd admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:45.784918 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a8374-18d6-4a14-a2b6-cd38da77f934 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:45.787713 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a8374-18d6-4a14-a2b6-cd38da77f934 service cinder] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:45.802918 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1015/2040] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:54:45 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:54:45.806699 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d67f0d-0618-44c8-92fa-317f0569ba52 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:45.807045 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d67f0d-0618-44c8-92fa-317f0569ba52 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:45.807420 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d67f0d-0618-44c8-92fa-317f0569ba52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:45.832690 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-57d67f0d-0618-44c8-92fa-317f0569ba52 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:45.834439 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-57d67f0d-0618-44c8-92fa-317f0569ba52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:45.842707 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1016/2041] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:54:45.847154 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d67f0d-0618-44c8-92fa-317f0569ba52 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:45.855402 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1026/2042] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:54:45 2022] POST /identity/v3/users => generated 298 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:54:45.857286 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d67f0d-0618-44c8-92fa-317f0569ba52 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:45.857959 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a437ba-57e7-423d-ab91-f84953075edd admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:45.867477 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571aeb5d-864f-429b-a0c2-01017903dcb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:45.868571 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571aeb5d-864f-429b-a0c2-01017903dcb4 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:45.868689 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a437ba-57e7-423d-ab91-f84953075edd admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:45.869093 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571aeb5d-864f-429b-a0c2-01017903dcb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:45.869512 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571aeb5d-864f-429b-a0c2-01017903dcb4 admin admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:45.870814 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571aeb5d-864f-429b-a0c2-01017903dcb4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:54:45.871328 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571aeb5d-864f-429b-a0c2-01017903dcb4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:45.873442 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571aeb5d-864f-429b-a0c2-01017903dcb4 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:45.874409 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2840d75a-1a61-4c97-a49e-46afaf1f0498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:45.875121 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2840d75a-1a61-4c97-a49e-46afaf1f0498 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:45.875410 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2840d75a-1a61-4c97-a49e-46afaf1f0498 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:45.875689 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2840d75a-1a61-4c97-a49e-46afaf1f0498 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:45.876726 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2840d75a-1a61-4c97-a49e-46afaf1f0498 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:45.878060 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2840d75a-1a61-4c97-a49e-46afaf1f0498 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:45.889182 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1027/2043] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:54:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:54:45.893019 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2840d75a-1a61-4c97-a49e-46afaf1f0498 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:45.895046 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1017/2044] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 22:54:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:54:45.899408 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571aeb5d-864f-429b-a0c2-01017903dcb4 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:45.906951 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2840d75a-1a61-4c97-a49e-46afaf1f0498 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:45.910214 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571aeb5d-864f-429b-a0c2-01017903dcb4 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:45.912028 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf3efc-322c-480b-82db-3d414e59031d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:45.912964 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf3efc-322c-480b-82db-3d414e59031d admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:45.913302 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf3efc-322c-480b-82db-3d414e59031d admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:45.913723 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf3efc-322c-480b-82db-3d414e59031d admin admin] PATH_INFO: `/v3/projects/4b8b773ffcab4a3cae7ef5232f763576/users/b3c6ad69c2f6497aac901a9f16996024/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:45.916866 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83c965a-68c4-47f2-a045-9f3494426a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:45.916866 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c965a-68c4-47f2-a045-9f3494426a2a admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:45.917402 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c965a-68c4-47f2-a045-9f3494426a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:45.917737 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c965a-68c4-47f2-a045-9f3494426a2a admin admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:45.918553 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83c965a-68c4-47f2-a045-9f3494426a2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:45.920682 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83c965a-68c4-47f2-a045-9f3494426a2a admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:45.930896 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cf3efc-322c-480b-82db-3d414e59031d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b8b773ffcab4a3cae7ef5232f763576, user_id=b3c6ad69c2f6497aac901a9f16996024, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:45.933606 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cf3efc-322c-480b-82db-3d414e59031d admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:46.013912 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1018/2045] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:54:45 2022] POST /identity/v3/projects => generated 383 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:54:46.014918 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83c965a-68c4-47f2-a045-9f3494426a2a admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:46.041184 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1028/2046] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:54:45 2022] PUT /identity/v3/projects/4b8b773ffcab4a3cae7ef5232f763576/users/b3c6ad69c2f6497aac901a9f16996024/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:54:46.046445 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf3efc-322c-480b-82db-3d414e59031d admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:46.085989 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:46.101235 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83c965a-68c4-47f2-a045-9f3494426a2a admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:46.131793 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf3efc-322c-480b-82db-3d414e59031d admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:46.137998 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56822508-2b98-4236-81d7-f6cdcc425013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:46.138675 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56822508-2b98-4236-81d7-f6cdcc425013 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:46.138979 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56822508-2b98-4236-81d7-f6cdcc425013 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:46.139386 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56822508-2b98-4236-81d7-f6cdcc425013 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:46.140443 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56822508-2b98-4236-81d7-f6cdcc425013 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:46.142049 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56822508-2b98-4236-81d7-f6cdcc425013 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:46.156326 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1029/2047] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:54:46 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:54:46.157948 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56822508-2b98-4236-81d7-f6cdcc425013 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:46.167824 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56822508-2b98-4236-81d7-f6cdcc425013 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:46.172860 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d22b445-fd81-47dc-a099-a8dc13cd3ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:46.176262 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d22b445-fd81-47dc-a099-a8dc13cd3ff5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:46.176262 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d22b445-fd81-47dc-a099-a8dc13cd3ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:46.176262 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d22b445-fd81-47dc-a099-a8dc13cd3ff5 admin admin] PATH_INFO: `/v3/projects/4b8b773ffcab4a3cae7ef5232f763576/users/b3c6ad69c2f6497aac901a9f16996024/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:46.176262 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d22b445-fd81-47dc-a099-a8dc13cd3ff5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b8b773ffcab4a3cae7ef5232f763576, user_id=b3c6ad69c2f6497aac901a9f16996024, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:46.176636 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6843ab3b-28d8-4c68-b81a-dd6e4b93b1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:46.177965 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843ab3b-28d8-4c68-b81a-dd6e4b93b1d3 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:46.178315 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843ab3b-28d8-4c68-b81a-dd6e4b93b1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:46.178671 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843ab3b-28d8-4c68-b81a-dd6e4b93b1d3 admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:46.179455 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6843ab3b-28d8-4c68-b81a-dd6e4b93b1d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:46.179571 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d22b445-fd81-47dc-a099-a8dc13cd3ff5 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:46.181431 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6843ab3b-28d8-4c68-b81a-dd6e4b93b1d3 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:46.190097 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1030/2048] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:54:46 2022] PUT /identity/v3/projects/4b8b773ffcab4a3cae7ef5232f763576/users/b3c6ad69c2f6497aac901a9f16996024/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:46.209785 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d22b445-fd81-47dc-a099-a8dc13cd3ff5 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:46.212674 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-2d22b445-fd81-47dc-a099-a8dc13cd3ff5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:46.286131 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d22b445-fd81-47dc-a099-a8dc13cd3ff5 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:46.291850 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddb310a-d483-41b1-90cc-72ea66118b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:46.292602 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb310a-d483-41b1-90cc-72ea66118b02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:46.292907 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb310a-d483-41b1-90cc-72ea66118b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:46.293166 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb310a-d483-41b1-90cc-72ea66118b02 admin admin] PATH_INFO: `/v3/projects/2e6a9a32d58d460c839eaeeabd890743` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:46.294674 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddb310a-d483-41b1-90cc-72ea66118b02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e6a9a32d58d460c839eaeeabd890743)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:46.296707 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddb310a-d483-41b1-90cc-72ea66118b02 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:46.309164 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1019/2049] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:54:46 2022] POST /identity/v3/users => generated 292 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:54:46.313697 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0803e7-4bbc-486f-b9d0-61a77f1ceb07 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:46.316753 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0803e7-4bbc-486f-b9d0-61a77f1ceb07 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:46.317120 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0803e7-4bbc-486f-b9d0-61a77f1ceb07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:46.326438 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:46.357405 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-ea0803e7-4bbc-486f-b9d0-61a77f1ceb07 None None] MFA Rules not processed for user `b3c6ad69c2f6497aac901a9f16996024`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:46.359181 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-ea0803e7-4bbc-486f-b9d0-61a77f1ceb07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:46.400489 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-5ddb310a-d483-41b1-90cc-72ea66118b02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e6a9a32d58d460c839eaeeabd890743', 'request_id': 'req-5ddb310a-d483-41b1-90cc-72ea66118b02'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:46.406017 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-5ddb310a-d483-41b1-90cc-72ea66118b02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e6a9a32d58d460c839eaeeabd890743', 'request_id': 'req-5ddb310a-d483-41b1-90cc-72ea66118b02'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:46.428216 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1020/2050] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:46 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:54:46.429055 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0803e7-4bbc-486f-b9d0-61a77f1ceb07 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:46.439341 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0803e7-4bbc-486f-b9d0-61a77f1ceb07 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:46.496733 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4bff85-adac-4689-b4f9-a6605d6bfc13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:46.497426 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4bff85-adac-4689-b4f9-a6605d6bfc13 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:46.497856 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4bff85-adac-4689-b4f9-a6605d6bfc13 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:46.498132 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4bff85-adac-4689-b4f9-a6605d6bfc13 admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:46.498909 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4bff85-adac-4689-b4f9-a6605d6bfc13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:46.500508 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4bff85-adac-4689-b4f9-a6605d6bfc13 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:46.510750 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1021/2051] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:54:46 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:54:46.511577 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4bff85-adac-4689-b4f9-a6605d6bfc13 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:46.523088 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-5ddb310a-d483-41b1-90cc-72ea66118b02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e6a9a32d58d460c839eaeeabd890743', 'request_id': 'req-5ddb310a-d483-41b1-90cc-72ea66118b02'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:46.537033 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-5ddb310a-d483-41b1-90cc-72ea66118b02 admin admin] The token cache is being invalidate because project 2e6a9a32d58d460c839eaeeabd890743 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:54:46.537287 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-5ddb310a-d483-41b1-90cc-72ea66118b02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:46.537989 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-5ddb310a-d483-41b1-90cc-72ea66118b02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:46.539911 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1031/2052] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:54:46 2022] DELETE /identity/v3/projects/2e6a9a32d58d460c839eaeeabd890743 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:46.540895 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddb310a-d483-41b1-90cc-72ea66118b02 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:46.542477 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-5ddb310a-d483-41b1-90cc-72ea66118b02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:46.560134 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddb310a-d483-41b1-90cc-72ea66118b02 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:46.565350 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33042d7a-e21c-47f8-bfde-bab22491fd86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:46.566021 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33042d7a-e21c-47f8-bfde-bab22491fd86 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:46.566308 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33042d7a-e21c-47f8-bfde-bab22491fd86 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:46.566592 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33042d7a-e21c-47f8-bfde-bab22491fd86 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:46.567509 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33042d7a-e21c-47f8-bfde-bab22491fd86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:46.569063 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33042d7a-e21c-47f8-bfde-bab22491fd86 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:46.578475 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4bff85-adac-4689-b4f9-a6605d6bfc13 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:46.585550 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1032/2053] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:54:46 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:54:46.585648 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88775ced-66dd-458c-82cd-9c6d7e173478 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:46.585738 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e4dfa5-bd24-4fdb-8f12-ad04270eae45 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:46.585738 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e4dfa5-bd24-4fdb-8f12-ad04270eae45 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:46.585880 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88775ced-66dd-458c-82cd-9c6d7e173478 service cinder] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:46.585963 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e4dfa5-bd24-4fdb-8f12-ad04270eae45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:46.586045 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88775ced-66dd-458c-82cd-9c6d7e173478 service cinder] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:46.586045 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88775ced-66dd-458c-82cd-9c6d7e173478 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:46.586045 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-88775ced-66dd-458c-82cd-9c6d7e173478 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:46.611450 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-f1e4dfa5-bd24-4fdb-8f12-ad04270eae45 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:46.616032 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-f1e4dfa5-bd24-4fdb-8f12-ad04270eae45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:46.623297 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1033/2054] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:54:46.623740 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e4dfa5-bd24-4fdb-8f12-ad04270eae45 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:46.638086 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e4dfa5-bd24-4fdb-8f12-ad04270eae45 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:46.646873 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb7031b-4752-4913-9f3a-3925b04c930f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:46.646873 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb7031b-4752-4913-9f3a-3925b04c930f admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:46.646873 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb7031b-4752-4913-9f3a-3925b04c930f admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:46.646873 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb7031b-4752-4913-9f3a-3925b04c930f admin admin] PATH_INFO: `/v3/projects/224678a5f1db4416b8175d315a4e4a0d/users/8c11dc2f4c764dc58458d9226efc878e/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:46.657399 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88775ced-66dd-458c-82cd-9c6d7e173478 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:46.659404 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88775ced-66dd-458c-82cd-9c6d7e173478 service cinder] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:46.660169 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb7031b-4752-4913-9f3a-3925b04c930f admin admin] RBAC: Authorizing `identity:create_grant(project_id=224678a5f1db4416b8175d315a4e4a0d, user_id=8c11dc2f4c764dc58458d9226efc878e, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:46.663423 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb7031b-4752-4913-9f3a-3925b04c930f admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:46.670137 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1022/2055] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:54:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:54:46.671686 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7548a7-0a87-409c-8ee2-a82bcae90aae None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:46.671981 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7548a7-0a87-409c-8ee2-a82bcae90aae None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:46.672262 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7548a7-0a87-409c-8ee2-a82bcae90aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:46.691220 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-5d7548a7-0a87-409c-8ee2-a82bcae90aae None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:46.692456 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-5d7548a7-0a87-409c-8ee2-a82bcae90aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:46.700818 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1023/2056] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:54:46.705280 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7548a7-0a87-409c-8ee2-a82bcae90aae None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:46.715847 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7548a7-0a87-409c-8ee2-a82bcae90aae None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:46.720983 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14214fb-5495-40fb-b73b-9592ede7d2c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:46.721697 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14214fb-5495-40fb-b73b-9592ede7d2c8 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:46.722063 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14214fb-5495-40fb-b73b-9592ede7d2c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:46.722346 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14214fb-5495-40fb-b73b-9592ede7d2c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:46.723132 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14214fb-5495-40fb-b73b-9592ede7d2c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:54:46.723614 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14214fb-5495-40fb-b73b-9592ede7d2c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:46.725912 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14214fb-5495-40fb-b73b-9592ede7d2c8 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:46.745145 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1024/2057] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 22:54:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:54:46.746107 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14214fb-5495-40fb-b73b-9592ede7d2c8 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:46.748494 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1034/2058] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:54:46 2022] PUT /identity/v3/projects/224678a5f1db4416b8175d315a4e4a0d/users/8c11dc2f4c764dc58458d9226efc878e/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:46.753922 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74bad3a-5e8b-44cc-adcb-bce6fce82258 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:46.755110 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74bad3a-5e8b-44cc-adcb-bce6fce82258 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:46.756132 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74bad3a-5e8b-44cc-adcb-bce6fce82258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:46.763257 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14214fb-5495-40fb-b73b-9592ede7d2c8 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:46.782351 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-f74bad3a-5e8b-44cc-adcb-bce6fce82258 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:46.784198 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-f74bad3a-5e8b-44cc-adcb-bce6fce82258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:46.836688 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aa1bd0-0a38-4a69-a02e-5d333ac3be58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:46.837373 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa1bd0-0a38-4a69-a02e-5d333ac3be58 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:46.838119 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa1bd0-0a38-4a69-a02e-5d333ac3be58 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:46.838391 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa1bd0-0a38-4a69-a02e-5d333ac3be58 admin admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:46.839523 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aa1bd0-0a38-4a69-a02e-5d333ac3be58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:46.842180 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:46.844309 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aa1bd0-0a38-4a69-a02e-5d333ac3be58 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:46.854148 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1035/2059] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:54:46.868156 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74bad3a-5e8b-44cc-adcb-bce6fce82258 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:46.869707 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-f74bad3a-5e8b-44cc-adcb-bce6fce82258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:46.930572 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74bad3a-5e8b-44cc-adcb-bce6fce82258 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:46.933192 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1025/2060] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:54:46 2022] POST /identity/v3/projects => generated 373 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:54:46.934032 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aa1bd0-0a38-4a69-a02e-5d333ac3be58 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:46.937265 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-43aa1bd0-0a38-4a69-a02e-5d333ac3be58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:46.996583 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79777b31-5474-4109-bcdb-bb1324c47e64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:46.997526 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79777b31-5474-4109-bcdb-bb1324c47e64 service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:46.998140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79777b31-5474-4109-bcdb-bb1324c47e64 service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:46.998429 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79777b31-5474-4109-bcdb-bb1324c47e64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:47.016159 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aa1bd0-0a38-4a69-a02e-5d333ac3be58 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:47.025692 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded4ae53-adab-4404-b7fc-4abda9c7c3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:47.027156 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded4ae53-adab-4404-b7fc-4abda9c7c3c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:47.027625 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded4ae53-adab-4404-b7fc-4abda9c7c3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:47.027985 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded4ae53-adab-4404-b7fc-4abda9c7c3c5 admin admin] PATH_INFO: `/v3/users/c4b3ac2700db452f9973978b8b97ac70` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:47.029697 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded4ae53-adab-4404-b7fc-4abda9c7c3c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4b3ac2700db452f9973978b8b97ac70)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:47.032231 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded4ae53-adab-4404-b7fc-4abda9c7c3c5 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:47.068009 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79777b31-5474-4109-bcdb-bb1324c47e64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:47.071634 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79777b31-5474-4109-bcdb-bb1324c47e64 service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:47.079290 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1036/2061] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:54:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:54:47.080162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79777b31-5474-4109-bcdb-bb1324c47e64 service neutron] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:47.089598 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79777b31-5474-4109-bcdb-bb1324c47e64 service neutron] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:47.095961 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b0280-25b0-443a-ba26-aa9ea0cfa773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:47.096797 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b0280-25b0-443a-ba26-aa9ea0cfa773 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:47.097156 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b0280-25b0-443a-ba26-aa9ea0cfa773 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:47.097508 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b0280-25b0-443a-ba26-aa9ea0cfa773 admin admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:47.098282 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932b0280-25b0-443a-ba26-aa9ea0cfa773 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:47.101134 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932b0280-25b0-443a-ba26-aa9ea0cfa773 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:47.227896 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1037/2062] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:54:47 2022] POST /identity/v3/users => generated 287 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:54:47.231988 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b0280-25b0-443a-ba26-aa9ea0cfa773 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:47.234318 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-ded4ae53-adab-4404-b7fc-4abda9c7c3c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4b3ac2700db452f9973978b8b97ac70'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:47.246021 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-ded4ae53-adab-4404-b7fc-4abda9c7c3c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4b3ac2700db452f9973978b8b97ac70'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:47.247153 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-ded4ae53-adab-4404-b7fc-4abda9c7c3c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4b3ac2700db452f9973978b8b97ac70'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:47.247725 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b0280-25b0-443a-ba26-aa9ea0cfa773 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:47.256125 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d16f25-669e-431f-82d6-f33928cb31f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:47.256849 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d16f25-669e-431f-82d6-f33928cb31f5 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:47.257443 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d16f25-669e-431f-82d6-f33928cb31f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:47.257443 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d16f25-669e-431f-82d6-f33928cb31f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:47.258201 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d16f25-669e-431f-82d6-f33928cb31f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:47.259912 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d16f25-669e-431f-82d6-f33928cb31f5 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:47.265784 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-ded4ae53-adab-4404-b7fc-4abda9c7c3c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4b3ac2700db452f9973978b8b97ac70'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:47.266509 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-ded4ae53-adab-4404-b7fc-4abda9c7c3c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4b3ac2700db452f9973978b8b97ac70'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:47.274214 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1038/2063] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:54:47 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:54:47.278341 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d16f25-669e-431f-82d6-f33928cb31f5 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:47.280181 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-44d16f25-669e-431f-82d6-f33928cb31f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:47.293741 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d16f25-669e-431f-82d6-f33928cb31f5 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:47.298464 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6657abf-3c5b-4071-9c75-0fb8729569e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:47.299170 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6657abf-3c5b-4071-9c75-0fb8729569e0 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:47.299461 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6657abf-3c5b-4071-9c75-0fb8729569e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:47.299742 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6657abf-3c5b-4071-9c75-0fb8729569e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:47.300612 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6657abf-3c5b-4071-9c75-0fb8729569e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:47.302765 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6657abf-3c5b-4071-9c75-0fb8729569e0 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:47.313400 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1039/2064] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:54:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:54:47.317524 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6657abf-3c5b-4071-9c75-0fb8729569e0 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:47.326736 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6657abf-3c5b-4071-9c75-0fb8729569e0 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:47.331191 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d28e11-48a4-4ebb-ba82-c427704bac39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:47.332020 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d28e11-48a4-4ebb-ba82-c427704bac39 admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:47.332298 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d28e11-48a4-4ebb-ba82-c427704bac39 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:47.332571 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d28e11-48a4-4ebb-ba82-c427704bac39 admin admin] PATH_INFO: `/v3/projects/93d60e3f3bcd48f292f30a8d2f0351d7/users/a0a765202a9645a1acc332f559537458/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:47.349323 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:47.349548 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d28e11-48a4-4ebb-ba82-c427704bac39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93d60e3f3bcd48f292f30a8d2f0351d7, user_id=a0a765202a9645a1acc332f559537458, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:47.352169 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d28e11-48a4-4ebb-ba82-c427704bac39 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:47.404651 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1026/2065] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:54:46 2022] DELETE /identity/v3/users/c4b3ac2700db452f9973978b8b97ac70 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:47.414716 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507012d5-ad16-4a4f-aa0c-ce43710d1f4b None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:47.414716 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507012d5-ad16-4a4f-aa0c-ce43710d1f4b None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:47.414716 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507012d5-ad16-4a4f-aa0c-ce43710d1f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:47.428647 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-507012d5-ad16-4a4f-aa0c-ce43710d1f4b None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:47.429928 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-507012d5-ad16-4a4f-aa0c-ce43710d1f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:47.445850 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1040/2066] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:54:47 2022] PUT /identity/v3/projects/93d60e3f3bcd48f292f30a8d2f0351d7/users/a0a765202a9645a1acc332f559537458/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:54:47.450634 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b59c22-6e1e-4dd9-8e77-d777b318c6fb None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:47.450994 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b59c22-6e1e-4dd9-8e77-d777b318c6fb None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:47.451309 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b59c22-6e1e-4dd9-8e77-d777b318c6fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:47.478757 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-54b59c22-6e1e-4dd9-8e77-d777b318c6fb None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:47.490970 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-54b59c22-6e1e-4dd9-8e77-d777b318c6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:47.530876 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1027/2067] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:54:47.542594 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507012d5-ad16-4a4f-aa0c-ce43710d1f4b None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:47.547977 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-507012d5-ad16-4a4f-aa0c-ce43710d1f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:47.559297 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1041/2068] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:54:47.573193 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b59c22-6e1e-4dd9-8e77-d777b318c6fb None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:47.574591 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-54b59c22-6e1e-4dd9-8e77-d777b318c6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:47.614802 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507012d5-ad16-4a4f-aa0c-ce43710d1f4b None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:47.655364 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b59c22-6e1e-4dd9-8e77-d777b318c6fb None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:47.659215 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0241fe4-028f-46d4-921d-63b4593a1bc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:47.659983 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0241fe4-028f-46d4-921d-63b4593a1bc5 service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:47.660430 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0241fe4-028f-46d4-921d-63b4593a1bc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:47.660777 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0241fe4-028f-46d4-921d-63b4593a1bc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:47.671185 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0241fe4-028f-46d4-921d-63b4593a1bc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:47.672938 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0241fe4-028f-46d4-921d-63b4593a1bc5 service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:47.680081 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a9cae9-0708-4c39-8e20-1f9a0980923f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:47.683915 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a9cae9-0708-4c39-8e20-1f9a0980923f service neutron] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:47.683915 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a9cae9-0708-4c39-8e20-1f9a0980923f service neutron] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:47.683915 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a9cae9-0708-4c39-8e20-1f9a0980923f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:47.684536 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1042/2069] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:54:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:54:47.698803 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a9cae9-0708-4c39-8e20-1f9a0980923f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:47.700309 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a9cae9-0708-4c39-8e20-1f9a0980923f service neutron] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:47.712767 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1028/2070] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:54:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:54:48.131131 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:48.199951 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:48.244626 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0241fe4-028f-46d4-921d-63b4593a1bc5 service neutron] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:48.246092 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-e0241fe4-028f-46d4-921d-63b4593a1bc5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:48.258787 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0241fe4-028f-46d4-921d-63b4593a1bc5 service neutron] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:48.263783 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117750fb-b850-4d97-bf50-9e5621630746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:48.264794 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117750fb-b850-4d97-bf50-9e5621630746 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:48.265209 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117750fb-b850-4d97-bf50-9e5621630746 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:48.265562 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117750fb-b850-4d97-bf50-9e5621630746 admin admin] PATH_INFO: `/v3/projects/f3747675d8d34576ad27b1eff96aabcc` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:48.267807 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117750fb-b850-4d97-bf50-9e5621630746 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3747675d8d34576ad27b1eff96aabcc)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:48.271451 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117750fb-b850-4d97-bf50-9e5621630746 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:48.387061 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-117750fb-b850-4d97-bf50-9e5621630746 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3747675d8d34576ad27b1eff96aabcc', 'request_id': 'req-117750fb-b850-4d97-bf50-9e5621630746'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:48.392584 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-117750fb-b850-4d97-bf50-9e5621630746 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3747675d8d34576ad27b1eff96aabcc', 'request_id': 'req-117750fb-b850-4d97-bf50-9e5621630746'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:48.479437 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-117750fb-b850-4d97-bf50-9e5621630746 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3747675d8d34576ad27b1eff96aabcc', 'request_id': 'req-117750fb-b850-4d97-bf50-9e5621630746'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:48.495440 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-117750fb-b850-4d97-bf50-9e5621630746 admin admin] The token cache is being invalidate because project f3747675d8d34576ad27b1eff96aabcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:54:48.496017 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-117750fb-b850-4d97-bf50-9e5621630746 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:48.496739 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-117750fb-b850-4d97-bf50-9e5621630746 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:48.501547 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1043/2071] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:54:48 2022] DELETE /identity/v3/projects/f3747675d8d34576ad27b1eff96aabcc => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:48.506212 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a9cae9-0708-4c39-8e20-1f9a0980923f service neutron] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:48.516509 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a9cae9-0708-4c39-8e20-1f9a0980923f service neutron] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:48.572158 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:48.577567 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9755bd3-6dda-4000-81dd-81dbcea2c5f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:48.578590 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9755bd3-6dda-4000-81dd-81dbcea2c5f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:48.581810 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9755bd3-6dda-4000-81dd-81dbcea2c5f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:48.581810 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9755bd3-6dda-4000-81dd-81dbcea2c5f6 admin admin] PATH_INFO: `/v3/users/89082c9061ef4fabba5869845d348488` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:48.584209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9755bd3-6dda-4000-81dd-81dbcea2c5f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89082c9061ef4fabba5869845d348488)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:48.586300 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9755bd3-6dda-4000-81dd-81dbcea2c5f6 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:48.809501 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-f9755bd3-6dda-4000-81dd-81dbcea2c5f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89082c9061ef4fabba5869845d348488'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:48.819872 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-f9755bd3-6dda-4000-81dd-81dbcea2c5f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89082c9061ef4fabba5869845d348488'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:48.820645 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-f9755bd3-6dda-4000-81dd-81dbcea2c5f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89082c9061ef4fabba5869845d348488'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:48.862877 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-f9755bd3-6dda-4000-81dd-81dbcea2c5f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89082c9061ef4fabba5869845d348488'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:48.865041 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-f9755bd3-6dda-4000-81dd-81dbcea2c5f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89082c9061ef4fabba5869845d348488'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:48.885714 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117750fb-b850-4d97-bf50-9e5621630746 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:48.887202 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-117750fb-b850-4d97-bf50-9e5621630746 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:48.954610 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117750fb-b850-4d97-bf50-9e5621630746 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:48.959518 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bd57be-d6aa-4f5c-b26b-04e97269b8b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:48.960286 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd57be-d6aa-4f5c-b26b-04e97269b8b4 service cinder] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:48.960572 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd57be-d6aa-4f5c-b26b-04e97269b8b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:48.965148 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd57be-d6aa-4f5c-b26b-04e97269b8b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:48.965148 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-66bd57be-d6aa-4f5c-b26b-04e97269b8b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:49.009783 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1029/2072] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:54:48 2022] DELETE /identity/v3/users/89082c9061ef4fabba5869845d348488 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:54:49.030818 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bd57be-d6aa-4f5c-b26b-04e97269b8b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:49.032895 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bd57be-d6aa-4f5c-b26b-04e97269b8b4 service cinder] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:49.042436 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1044/2073] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:54:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:54:49.588767 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9755bd3-6dda-4000-81dd-81dbcea2c5f6 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:49.591320 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-f9755bd3-6dda-4000-81dd-81dbcea2c5f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:49.605386 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:49.713719 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9755bd3-6dda-4000-81dd-81dbcea2c5f6 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:49.721723 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5937f2cd-176e-456c-99e0-c21c43d82c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:49.722941 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5937f2cd-176e-456c-99e0-c21c43d82c24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:49.723240 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5937f2cd-176e-456c-99e0-c21c43d82c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:49.724125 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5937f2cd-176e-456c-99e0-c21c43d82c24 admin admin] PATH_INFO: `/v3/projects/181f59b2ba7c4854b77624390f9ad1c4` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:49.725503 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5937f2cd-176e-456c-99e0-c21c43d82c24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=181f59b2ba7c4854b77624390f9ad1c4)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:49.727582 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5937f2cd-176e-456c-99e0-c21c43d82c24 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:49.766462 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:49.869524 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-5937f2cd-176e-456c-99e0-c21c43d82c24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '181f59b2ba7c4854b77624390f9ad1c4', 'request_id': 'req-5937f2cd-176e-456c-99e0-c21c43d82c24'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:49.883968 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-5937f2cd-176e-456c-99e0-c21c43d82c24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '181f59b2ba7c4854b77624390f9ad1c4', 'request_id': 'req-5937f2cd-176e-456c-99e0-c21c43d82c24'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:49.993535 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-5937f2cd-176e-456c-99e0-c21c43d82c24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '181f59b2ba7c4854b77624390f9ad1c4', 'request_id': 'req-5937f2cd-176e-456c-99e0-c21c43d82c24'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:50.008488 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-5937f2cd-176e-456c-99e0-c21c43d82c24 admin admin] The token cache is being invalidate because project 181f59b2ba7c4854b77624390f9ad1c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:54:50.008782 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-5937f2cd-176e-456c-99e0-c21c43d82c24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:50.009519 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-5937f2cd-176e-456c-99e0-c21c43d82c24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:50.014543 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1030/2074] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:54:49 2022] DELETE /identity/v3/projects/181f59b2ba7c4854b77624390f9ad1c4 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:50.016268 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587d026-96b6-4c3c-afef-a5c5a4ed27ce None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:50.016618 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587d026-96b6-4c3c-afef-a5c5a4ed27ce None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:50.016870 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587d026-96b6-4c3c-afef-a5c5a4ed27ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:50.039250 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-3587d026-96b6-4c3c-afef-a5c5a4ed27ce None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:50.040783 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-3587d026-96b6-4c3c-afef-a5c5a4ed27ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:50.178211 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1045/2075] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:54:50.187102 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81908dc1-9dde-4fc0-9042-4a102ffe2f92 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:50.187477 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81908dc1-9dde-4fc0-9042-4a102ffe2f92 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:50.187870 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81908dc1-9dde-4fc0-9042-4a102ffe2f92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:50.209997 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-81908dc1-9dde-4fc0-9042-4a102ffe2f92 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:50.211425 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-81908dc1-9dde-4fc0-9042-4a102ffe2f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:50.228145 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3587d026-96b6-4c3c-afef-a5c5a4ed27ce None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:50.236023 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1031/2076] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:54:50.240424 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3587d026-96b6-4c3c-afef-a5c5a4ed27ce None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:50.245606 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e817d82-4069-4fbb-9b75-ebd60c4f556d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:50.246361 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e817d82-4069-4fbb-9b75-ebd60c4f556d admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:50.246750 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e817d82-4069-4fbb-9b75-ebd60c4f556d admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:50.247133 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e817d82-4069-4fbb-9b75-ebd60c4f556d admin admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:50.248027 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e817d82-4069-4fbb-9b75-ebd60c4f556d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:54:50.248398 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e817d82-4069-4fbb-9b75-ebd60c4f556d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:50.250069 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e817d82-4069-4fbb-9b75-ebd60c4f556d admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:50.268374 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81908dc1-9dde-4fc0-9042-4a102ffe2f92 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:50.276081 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1046/2077] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 22:54:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:54:50.280362 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81908dc1-9dde-4fc0-9042-4a102ffe2f92 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:50.285643 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c853c3eb-b38f-4a1e-a8d4-41f8e4ab5e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:50.287099 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c853c3eb-b38f-4a1e-a8d4-41f8e4ab5e80 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:50.288573 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c853c3eb-b38f-4a1e-a8d4-41f8e4ab5e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:50.288573 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c853c3eb-b38f-4a1e-a8d4-41f8e4ab5e80 admin admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:50.288573 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c853c3eb-b38f-4a1e-a8d4-41f8e4ab5e80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:50.290662 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c853c3eb-b38f-4a1e-a8d4-41f8e4ab5e80 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:50.323171 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:50.376309 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1032/2078] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:54:50 2022] POST /identity/v3/projects => generated 363 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:54:50.379415 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e817d82-4069-4fbb-9b75-ebd60c4f556d admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:50.390689 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e817d82-4069-4fbb-9b75-ebd60c4f556d admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:50.458634 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547c99db-3b12-4834-bda9-1806b70780d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:50.459434 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c99db-3b12-4834-bda9-1806b70780d2 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:50.459823 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c99db-3b12-4834-bda9-1806b70780d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:50.460304 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c99db-3b12-4834-bda9-1806b70780d2 admin admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:50.461089 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547c99db-3b12-4834-bda9-1806b70780d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:50.463868 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547c99db-3b12-4834-bda9-1806b70780d2 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:50.578003 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1047/2079] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:54:50 2022] POST /identity/v3/users => generated 282 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:54:50.579573 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c853c3eb-b38f-4a1e-a8d4-41f8e4ab5e80 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:50.593010 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c853c3eb-b38f-4a1e-a8d4-41f8e4ab5e80 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:50.605641 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0a7508-7db9-4fde-af7b-d322ea9babb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:50.606540 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0a7508-7db9-4fde-af7b-d322ea9babb5 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:50.607031 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0a7508-7db9-4fde-af7b-d322ea9babb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:50.607405 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0a7508-7db9-4fde-af7b-d322ea9babb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:50.608250 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0a7508-7db9-4fde-af7b-d322ea9babb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:50.610345 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0a7508-7db9-4fde-af7b-d322ea9babb5 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:50.624897 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1033/2080] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:54:50 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:54:50.629227 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547c99db-3b12-4834-bda9-1806b70780d2 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:50.641528 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547c99db-3b12-4834-bda9-1806b70780d2 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:50.647346 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91a0432-4718-4e2e-9fc9-d37f52b126f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:50.649591 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a0432-4718-4e2e-9fc9-d37f52b126f6 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:50.649591 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a0432-4718-4e2e-9fc9-d37f52b126f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:50.649753 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a0432-4718-4e2e-9fc9-d37f52b126f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:50.650521 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91a0432-4718-4e2e-9fc9-d37f52b126f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:50.652574 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91a0432-4718-4e2e-9fc9-d37f52b126f6 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:50.674128 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0a7508-7db9-4fde-af7b-d322ea9babb5 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:50.684978 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1048/2081] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:54:50 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:54:50.690752 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0a7508-7db9-4fde-af7b-d322ea9babb5 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:50.707665 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f51600-c70c-4652-a93d-8e50005f9b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:50.707665 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f51600-c70c-4652-a93d-8e50005f9b9f admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:50.709559 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f51600-c70c-4652-a93d-8e50005f9b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:50.709852 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f51600-c70c-4652-a93d-8e50005f9b9f admin admin] PATH_INFO: `/v3/projects/bd00b813323b4a429b455e1b34360042/users/51ee6ca4f6084e948e0ed3e5f4ca26ec/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:50.752877 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f51600-c70c-4652-a93d-8e50005f9b9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd00b813323b4a429b455e1b34360042, user_id=51ee6ca4f6084e948e0ed3e5f4ca26ec, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:50.764705 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f51600-c70c-4652-a93d-8e50005f9b9f admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:50.827484 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91a0432-4718-4e2e-9fc9-d37f52b126f6 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:50.829461 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-b91a0432-4718-4e2e-9fc9-d37f52b126f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:50.880404 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1034/2082] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:54:50 2022] PUT /identity/v3/projects/bd00b813323b4a429b455e1b34360042/users/51ee6ca4f6084e948e0ed3e5f4ca26ec/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:50.892259 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b487fe48-79e9-4942-bcfb-95174f16dd02 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:50.892560 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b487fe48-79e9-4942-bcfb-95174f16dd02 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:50.892887 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b487fe48-79e9-4942-bcfb-95174f16dd02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:50.959322 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-b487fe48-79e9-4942-bcfb-95174f16dd02 None None] MFA Rules not processed for user `51ee6ca4f6084e948e0ed3e5f4ca26ec`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:50.965168 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-b487fe48-79e9-4942-bcfb-95174f16dd02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:51.017672 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91a0432-4718-4e2e-9fc9-d37f52b126f6 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:51.129554 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093dcd02-b395-4ac6-873b-1734c4cd3575 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:51.130794 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093dcd02-b395-4ac6-873b-1734c4cd3575 service cinder] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:51.131086 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093dcd02-b395-4ac6-873b-1734c4cd3575 service cinder] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:51.131367 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093dcd02-b395-4ac6-873b-1734c4cd3575 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:51.133579 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-093dcd02-b395-4ac6-873b-1734c4cd3575 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:51.144358 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1035/2083] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:50 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:54:51.151763 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b487fe48-79e9-4942-bcfb-95174f16dd02 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:51.257374 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:51.267444 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093dcd02-b395-4ac6-873b-1734c4cd3575 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:51.267917 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b487fe48-79e9-4942-bcfb-95174f16dd02 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:51.268923 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093dcd02-b395-4ac6-873b-1734c4cd3575 service cinder] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:51.274979 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b99a5b8-2497-4dc2-9e2d-6d82f280f224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:51.275915 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99a5b8-2497-4dc2-9e2d-6d82f280f224 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:51.276367 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99a5b8-2497-4dc2-9e2d-6d82f280f224 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:51.276813 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99a5b8-2497-4dc2-9e2d-6d82f280f224 admin admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:51.277972 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b99a5b8-2497-4dc2-9e2d-6d82f280f224 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:51.281585 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b99a5b8-2497-4dc2-9e2d-6d82f280f224 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:51.289677 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1049/2084] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:54:50 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:54:51.436705 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1036/2085] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:54:51 2022] POST /identity/v3/projects => generated 361 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:54:51.440993 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093dcd02-b395-4ac6-873b-1734c4cd3575 service cinder] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:51.456033 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093dcd02-b395-4ac6-873b-1734c4cd3575 service cinder] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:51.609909 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee19c6-7682-4caf-bfca-d1ed1557ad0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:51.610667 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee19c6-7682-4caf-bfca-d1ed1557ad0f admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:51.610984 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee19c6-7682-4caf-bfca-d1ed1557ad0f admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:51.611534 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee19c6-7682-4caf-bfca-d1ed1557ad0f admin admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:51.612258 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ee19c6-7682-4caf-bfca-d1ed1557ad0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:51.614402 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ee19c6-7682-4caf-bfca-d1ed1557ad0f admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:51.722227 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1050/2086] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:54:51 2022] POST /identity/v3/users => generated 287 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:54:51.723933 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b99a5b8-2497-4dc2-9e2d-6d82f280f224 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:51.736482 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b99a5b8-2497-4dc2-9e2d-6d82f280f224 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:51.743380 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37605cc4-0dc8-44db-8c99-95928bffaddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:51.744658 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37605cc4-0dc8-44db-8c99-95928bffaddb admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:51.745124 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37605cc4-0dc8-44db-8c99-95928bffaddb admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:51.745674 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37605cc4-0dc8-44db-8c99-95928bffaddb admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:51.746683 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37605cc4-0dc8-44db-8c99-95928bffaddb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:51.748619 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37605cc4-0dc8-44db-8c99-95928bffaddb admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:51.768393 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee19c6-7682-4caf-bfca-d1ed1557ad0f admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:51.769055 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1037/2087] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:54:51 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:54:51.803105 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee19c6-7682-4caf-bfca-d1ed1557ad0f admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:51.814483 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50710f2-831d-4338-9bfb-32e48cd707ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:51.815450 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50710f2-831d-4338-9bfb-32e48cd707ba admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:51.815737 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50710f2-831d-4338-9bfb-32e48cd707ba admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:51.816033 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50710f2-831d-4338-9bfb-32e48cd707ba admin admin] PATH_INFO: `/v3/projects/bbafea3dad654f12b4921efb1991474a/users/13a963066e3d4a208d3821f0114921c6/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:51.842204 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50710f2-831d-4338-9bfb-32e48cd707ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbafea3dad654f12b4921efb1991474a, user_id=13a963066e3d4a208d3821f0114921c6, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:51.845220 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50710f2-831d-4338-9bfb-32e48cd707ba admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:51.966299 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1051/2088] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:54:51 2022] PUT /identity/v3/projects/bbafea3dad654f12b4921efb1991474a/users/13a963066e3d4a208d3821f0114921c6/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:51.970609 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37605cc4-0dc8-44db-8c99-95928bffaddb admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:51.981288 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37605cc4-0dc8-44db-8c99-95928bffaddb admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:52.057233 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dfb4a0-ed14-4941-bad0-cdbc752d63d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:52.058123 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dfb4a0-ed14-4941-bad0-cdbc752d63d5 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:52.058627 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dfb4a0-ed14-4941-bad0-cdbc752d63d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:52.058958 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dfb4a0-ed14-4941-bad0-cdbc752d63d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:52.059797 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dfb4a0-ed14-4941-bad0-cdbc752d63d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:52.061633 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dfb4a0-ed14-4941-bad0-cdbc752d63d5 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:52.078133 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1038/2089] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:54:51 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:54:52.082666 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50710f2-831d-4338-9bfb-32e48cd707ba admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:52.094249 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50710f2-831d-4338-9bfb-32e48cd707ba admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:52.105380 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc14fe3b-6371-40f4-8eff-e81cad2a387d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:52.106427 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14fe3b-6371-40f4-8eff-e81cad2a387d admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:52.107036 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14fe3b-6371-40f4-8eff-e81cad2a387d admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:52.107528 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14fe3b-6371-40f4-8eff-e81cad2a387d admin admin] PATH_INFO: `/v3/projects/bbafea3dad654f12b4921efb1991474a/users/13a963066e3d4a208d3821f0114921c6/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:52.111597 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc14fe3b-6371-40f4-8eff-e81cad2a387d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbafea3dad654f12b4921efb1991474a, user_id=13a963066e3d4a208d3821f0114921c6, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:52.115992 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc14fe3b-6371-40f4-8eff-e81cad2a387d admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:52.139974 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1052/2090] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:54:52 2022] PUT /identity/v3/projects/bbafea3dad654f12b4921efb1991474a/users/13a963066e3d4a208d3821f0114921c6/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:52.141086 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f5065-d852-48cc-a8a2-55ea40c1fd8d None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:52.141379 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f5065-d852-48cc-a8a2-55ea40c1fd8d None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:52.141660 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f5065-d852-48cc-a8a2-55ea40c1fd8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:52.189203 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-0b8f5065-d852-48cc-a8a2-55ea40c1fd8d None None] MFA Rules not processed for user `13a963066e3d4a208d3821f0114921c6`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:52.190468 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-0b8f5065-d852-48cc-a8a2-55ea40c1fd8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:52.318465 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1039/2091] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:52 2022] POST /identity/v3/auth/tokens => generated 3295 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:54:52.332263 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc14fe3b-6371-40f4-8eff-e81cad2a387d admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:52.405888 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc14fe3b-6371-40f4-8eff-e81cad2a387d admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:52.410549 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d10d38-5b8f-42ee-be62-ef48c4905c48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:52.411496 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d10d38-5b8f-42ee-be62-ef48c4905c48 service cinder] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:52.411843 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d10d38-5b8f-42ee-be62-ef48c4905c48 service cinder] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:52.412181 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d10d38-5b8f-42ee-be62-ef48c4905c48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:52.421906 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d10d38-5b8f-42ee-be62-ef48c4905c48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:52.423498 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d10d38-5b8f-42ee-be62-ef48c4905c48 service cinder] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:52.435162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1053/2092] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:54:52 2022] GET /identity/v3/auth/tokens => generated 3295 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:54:52.580707 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:52.677343 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8f5065-d852-48cc-a8a2-55ea40c1fd8d None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:52.688514 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8f5065-d852-48cc-a8a2-55ea40c1fd8d None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:52.694810 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba115e0d-0637-40f7-9138-4af6fd33cabf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:52.695651 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba115e0d-0637-40f7-9138-4af6fd33cabf service cinder] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:52.695651 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba115e0d-0637-40f7-9138-4af6fd33cabf service cinder] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:52.695651 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba115e0d-0637-40f7-9138-4af6fd33cabf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:52.716386 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:52.766564 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba115e0d-0637-40f7-9138-4af6fd33cabf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:52.768565 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba115e0d-0637-40f7-9138-4af6fd33cabf service cinder] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:52.784781 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1040/2093] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:54:52 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:54:53.361558 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d10d38-5b8f-42ee-be62-ef48c4905c48 service cinder] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:53.363064 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-20d10d38-5b8f-42ee-be62-ef48c4905c48 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:53.532245 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d10d38-5b8f-42ee-be62-ef48c4905c48 service cinder] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:53.562361 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053c92b8-1c23-4284-9cfd-4a772b56943b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:53.570434 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053c92b8-1c23-4284-9cfd-4a772b56943b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:53.575565 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053c92b8-1c23-4284-9cfd-4a772b56943b admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:53.575565 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053c92b8-1c23-4284-9cfd-4a772b56943b admin admin] PATH_INFO: `/v3/users/dd8483bf6f1044bc80b33892d7964e22` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:53.575565 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053c92b8-1c23-4284-9cfd-4a772b56943b admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd8483bf6f1044bc80b33892d7964e22)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:53.581125 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053c92b8-1c23-4284-9cfd-4a772b56943b admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:53.806364 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-053c92b8-1c23-4284-9cfd-4a772b56943b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd8483bf6f1044bc80b33892d7964e22'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:53.833839 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-053c92b8-1c23-4284-9cfd-4a772b56943b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd8483bf6f1044bc80b33892d7964e22'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:53.834827 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-053c92b8-1c23-4284-9cfd-4a772b56943b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd8483bf6f1044bc80b33892d7964e22'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:53.850915 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-053c92b8-1c23-4284-9cfd-4a772b56943b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd8483bf6f1044bc80b33892d7964e22'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:53.952690 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-053c92b8-1c23-4284-9cfd-4a772b56943b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd8483bf6f1044bc80b33892d7964e22'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:53.962962 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1054/2094] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:54:53 2022] DELETE /identity/v3/users/dd8483bf6f1044bc80b33892d7964e22 => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:53.968695 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b29a91-3c57-4503-872b-9998bee4b253 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:53.969184 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b29a91-3c57-4503-872b-9998bee4b253 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:53.969473 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b29a91-3c57-4503-872b-9998bee4b253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:54.005636 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-d0b29a91-3c57-4503-872b-9998bee4b253 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:54.005636 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-d0b29a91-3c57-4503-872b-9998bee4b253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:54.093223 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1041/2095] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:54:54.114115 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053c92b8-1c23-4284-9cfd-4a772b56943b admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:54.116049 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-053c92b8-1c23-4284-9cfd-4a772b56943b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:54.180993 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053c92b8-1c23-4284-9cfd-4a772b56943b admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:54.184767 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebac4de-6878-432d-bdc5-745cf627141a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:54.185417 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebac4de-6878-432d-bdc5-745cf627141a service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:54.185702 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebac4de-6878-432d-bdc5-745cf627141a service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:54.185980 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebac4de-6878-432d-bdc5-745cf627141a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:54.195091 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebac4de-6878-432d-bdc5-745cf627141a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:54.196510 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebac4de-6878-432d-bdc5-745cf627141a service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:54.206007 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1055/2096] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:54:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:54:54.331625 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:54.862746 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b29a91-3c57-4503-872b-9998bee4b253 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:54.864990 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-d0b29a91-3c57-4503-872b-9998bee4b253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:54.883574 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b29a91-3c57-4503-872b-9998bee4b253 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:54.894882 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b66a432-73e6-433c-9607-d2ebd6f55bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:54.895820 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66a432-73e6-433c-9607-d2ebd6f55bd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:54.896101 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66a432-73e6-433c-9607-d2ebd6f55bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:54.896379 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66a432-73e6-433c-9607-d2ebd6f55bd9 admin admin] PATH_INFO: `/v3/projects/b9325bcb8a5c429383a0a51f086ec437` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:54.901079 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b66a432-73e6-433c-9607-d2ebd6f55bd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9325bcb8a5c429383a0a51f086ec437)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:54.904201 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b66a432-73e6-433c-9607-d2ebd6f55bd9 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:55.101660 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-1b66a432-73e6-433c-9607-d2ebd6f55bd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9325bcb8a5c429383a0a51f086ec437', 'request_id': 'req-1b66a432-73e6-433c-9607-d2ebd6f55bd9'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:55.119865 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-1b66a432-73e6-433c-9607-d2ebd6f55bd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9325bcb8a5c429383a0a51f086ec437', 'request_id': 'req-1b66a432-73e6-433c-9607-d2ebd6f55bd9'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:55.216167 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-1b66a432-73e6-433c-9607-d2ebd6f55bd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9325bcb8a5c429383a0a51f086ec437', 'request_id': 'req-1b66a432-73e6-433c-9607-d2ebd6f55bd9'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:55.237284 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-1b66a432-73e6-433c-9607-d2ebd6f55bd9 admin admin] The token cache is being invalidate because project b9325bcb8a5c429383a0a51f086ec437 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:54:55.237727 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-1b66a432-73e6-433c-9607-d2ebd6f55bd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:55.238299 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-1b66a432-73e6-433c-9607-d2ebd6f55bd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:55.244380 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1042/2097] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:54:54 2022] DELETE /identity/v3/projects/b9325bcb8a5c429383a0a51f086ec437 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:55.252431 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f7e3c-2b09-4dca-b12c-374b7cb94c41 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:55.255366 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:55.263218 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f7e3c-2b09-4dca-b12c-374b7cb94c41 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:55.263218 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f7e3c-2b09-4dca-b12c-374b7cb94c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:55.303718 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-d28f7e3c-2b09-4dca-b12c-374b7cb94c41 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:55.303718 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-d28f7e3c-2b09-4dca-b12c-374b7cb94c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:55.452795 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1056/2098] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:54:55.460696 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db32c11d-626a-4547-806c-c4d6fc999e9a None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:55.464238 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db32c11d-626a-4547-806c-c4d6fc999e9a None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:55.464238 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db32c11d-626a-4547-806c-c4d6fc999e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:55.509824 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-db32c11d-626a-4547-806c-c4d6fc999e9a None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:55.511074 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-db32c11d-626a-4547-806c-c4d6fc999e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:55.544433 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1043/2099] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:54:55.548671 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28f7e3c-2b09-4dca-b12c-374b7cb94c41 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:55.572353 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28f7e3c-2b09-4dca-b12c-374b7cb94c41 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:55.613820 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c649e-3f74-4a31-afb1-923d75f94a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:55.615208 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c649e-3f74-4a31-afb1-923d75f94a19 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:55.615501 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c649e-3f74-4a31-afb1-923d75f94a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:55.615783 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c649e-3f74-4a31-afb1-923d75f94a19 admin admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:55.618723 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4c649e-3f74-4a31-afb1-923d75f94a19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:54:55.619255 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4c649e-3f74-4a31-afb1-923d75f94a19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:55.622083 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4c649e-3f74-4a31-afb1-923d75f94a19 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:55.653943 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1057/2100] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 22:54:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:54:55.656607 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db32c11d-626a-4547-806c-c4d6fc999e9a None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:55.671590 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db32c11d-626a-4547-806c-c4d6fc999e9a None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:55.680946 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c3c8c6-a406-4e25-846d-7322697968b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:55.681516 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3c8c6-a406-4e25-846d-7322697968b6 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:55.681882 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3c8c6-a406-4e25-846d-7322697968b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:55.682236 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3c8c6-a406-4e25-846d-7322697968b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:55.684161 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c3c8c6-a406-4e25-846d-7322697968b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:55.686388 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c3c8c6-a406-4e25-846d-7322697968b6 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:55.895998 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1044/2101] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:54:55 2022] POST /identity/v3/projects => generated 385 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:54:55.898525 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c649e-3f74-4a31-afb1-923d75f94a19 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:55.952705 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c649e-3f74-4a31-afb1-923d75f94a19 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:56.013183 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4531b-3d20-452a-ab9b-00a20005ff81 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:56.013183 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4531b-3d20-452a-ab9b-00a20005ff81 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:56.013336 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4531b-3d20-452a-ab9b-00a20005ff81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:56.056674 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-48a4531b-3d20-452a-ab9b-00a20005ff81 None None] MFA Rules not processed for user `8c11dc2f4c764dc58458d9226efc878e`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:56.059769 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-48a4531b-3d20-452a-ab9b-00a20005ff81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:56.069367 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da0a5c-af76-4c35-9b85-e23afffb9847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:56.070428 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da0a5c-af76-4c35-9b85-e23afffb9847 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:56.070994 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da0a5c-af76-4c35-9b85-e23afffb9847 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:56.071568 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da0a5c-af76-4c35-9b85-e23afffb9847 admin admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:56.072584 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da0a5c-af76-4c35-9b85-e23afffb9847 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:56.075299 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da0a5c-af76-4c35-9b85-e23afffb9847 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:56.163108 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1045/2102] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:54:56.180781 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a4531b-3d20-452a-ab9b-00a20005ff81 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:56.182268 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-48a4531b-3d20-452a-ab9b-00a20005ff81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:56.189127 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1058/2103] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:54:55 2022] POST /identity/v3/users => generated 293 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:54:56.190105 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da0a5c-af76-4c35-9b85-e23afffb9847 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:56.204539 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da0a5c-af76-4c35-9b85-e23afffb9847 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:56.229254 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8383e6e3-7560-45f2-8f29-98fc1a433838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:56.229940 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8383e6e3-7560-45f2-8f29-98fc1a433838 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:56.230223 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8383e6e3-7560-45f2-8f29-98fc1a433838 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:56.230497 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8383e6e3-7560-45f2-8f29-98fc1a433838 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:56.231261 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8383e6e3-7560-45f2-8f29-98fc1a433838 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:56.234079 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8383e6e3-7560-45f2-8f29-98fc1a433838 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:56.250571 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1059/2104] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:54:56.255033 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8383e6e3-7560-45f2-8f29-98fc1a433838 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:56.271005 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8383e6e3-7560-45f2-8f29-98fc1a433838 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:56.276515 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca42407-98d6-438c-95ce-ee4e6c9ac039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:56.280012 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca42407-98d6-438c-95ce-ee4e6c9ac039 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:56.280012 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca42407-98d6-438c-95ce-ee4e6c9ac039 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:56.280012 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca42407-98d6-438c-95ce-ee4e6c9ac039 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:56.280012 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca42407-98d6-438c-95ce-ee4e6c9ac039 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:56.281330 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca42407-98d6-438c-95ce-ee4e6c9ac039 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:56.296702 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1060/2105] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:54:56.302705 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca42407-98d6-438c-95ce-ee4e6c9ac039 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:56.314820 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca42407-98d6-438c-95ce-ee4e6c9ac039 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:56.322291 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c26876a-dc66-429c-a1b6-24128d2d09a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:56.323472 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26876a-dc66-429c-a1b6-24128d2d09a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:56.323901 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26876a-dc66-429c-a1b6-24128d2d09a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:56.324321 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26876a-dc66-429c-a1b6-24128d2d09a3 admin admin] PATH_INFO: `/v3/projects/40d62acc2a1a40aaa2a5cb7b4ef9072b/users/87ea5ca395f142bfbea27f7e24c7e56c/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:56.358381 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c26876a-dc66-429c-a1b6-24128d2d09a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40d62acc2a1a40aaa2a5cb7b4ef9072b, user_id=87ea5ca395f142bfbea27f7e24c7e56c, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:56.361668 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c26876a-dc66-429c-a1b6-24128d2d09a3 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:56.363695 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a4531b-3d20-452a-ab9b-00a20005ff81 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:56.372698 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7e34d3-5951-47e7-b9a6-ccd3099b4967 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bfdb655a2a44b33ab724daaf73a294b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:56.373885 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e34d3-5951-47e7-b9a6-ccd3099b4967 service nova] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:56.374559 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e34d3-5951-47e7-b9a6-ccd3099b4967 service nova] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:56.375390 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e34d3-5951-47e7-b9a6-ccd3099b4967 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:56.388111 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7e34d3-5951-47e7-b9a6-ccd3099b4967 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:56.389926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7e34d3-5951-47e7-b9a6-ccd3099b4967 service nova] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:56.409285 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1046/2106] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:54:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:54:56.455197 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1061/2107] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:54:56 2022] PUT /identity/v3/projects/40d62acc2a1a40aaa2a5cb7b4ef9072b/users/87ea5ca395f142bfbea27f7e24c7e56c/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:56.464123 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8931070d-1456-41e2-90e0-3bbce25d41f8 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:56.464643 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8931070d-1456-41e2-90e0-3bbce25d41f8 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:56.486583 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8931070d-1456-41e2-90e0-3bbce25d41f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:56.522566 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-8931070d-1456-41e2-90e0-3bbce25d41f8 None None] MFA Rules not processed for user `87ea5ca395f142bfbea27f7e24c7e56c`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:56.524426 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-8931070d-1456-41e2-90e0-3bbce25d41f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:56.636649 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1047/2108] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:54:56.658584 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c26876a-dc66-429c-a1b6-24128d2d09a3 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:56.660573 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-3c26876a-dc66-429c-a1b6-24128d2d09a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:56.774818 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c26876a-dc66-429c-a1b6-24128d2d09a3 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:56.795005 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e325920-80c2-44f2-8a77-ff0732753c4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:56.795005 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e325920-80c2-44f2-8a77-ff0732753c4d service cinder] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:56.795005 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e325920-80c2-44f2-8a77-ff0732753c4d service cinder] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:56.795005 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e325920-80c2-44f2-8a77-ff0732753c4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:56.800351 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6211a7-5af0-40f4-8d0b-0218356b0752 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:56.800646 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6211a7-5af0-40f4-8d0b-0218356b0752 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:56.800928 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6211a7-5af0-40f4-8d0b-0218356b0752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:56.806666 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e325920-80c2-44f2-8a77-ff0732753c4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:56.808509 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e325920-80c2-44f2-8a77-ff0732753c4d service cinder] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:56.820395 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1062/2109] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:54:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:54:56.853511 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-3a6211a7-5af0-40f4-8d0b-0218356b0752 None None] MFA Rules not processed for user `a0a765202a9645a1acc332f559537458`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:56.856143 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-3a6211a7-5af0-40f4-8d0b-0218356b0752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:56.969404 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1048/2110] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:54:56.973581 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e325920-80c2-44f2-8a77-ff0732753c4d service cinder] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:56.975497 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-7e325920-80c2-44f2-8a77-ff0732753c4d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:57.052187 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e325920-80c2-44f2-8a77-ff0732753c4d service cinder] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:57.058112 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d7d1bf-7683-4a7c-96a2-440e8cea25bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:57.059494 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7d1bf-7683-4a7c-96a2-440e8cea25bb admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:57.059782 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7d1bf-7683-4a7c-96a2-440e8cea25bb admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:57.060241 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7d1bf-7683-4a7c-96a2-440e8cea25bb admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:57.061163 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d7d1bf-7683-4a7c-96a2-440e8cea25bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:57.063118 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d7d1bf-7683-4a7c-96a2-440e8cea25bb admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:57.152356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1063/2111] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:54:56 2022] POST /identity/v3/projects => generated 371 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:54:57.156616 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6211a7-5af0-40f4-8d0b-0218356b0752 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:57.172553 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6211a7-5af0-40f4-8d0b-0218356b0752 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:57.267846 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50c2054-fe24-41aa-90c1-67f0a6d4cd35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:57.268601 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c2054-fe24-41aa-90c1-67f0a6d4cd35 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:57.268937 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c2054-fe24-41aa-90c1-67f0a6d4cd35 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:57.269222 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c2054-fe24-41aa-90c1-67f0a6d4cd35 admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:57.270517 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50c2054-fe24-41aa-90c1-67f0a6d4cd35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:57.272715 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50c2054-fe24-41aa-90c1-67f0a6d4cd35 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:57.342390 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:57.412376 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1049/2112] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:54:57 2022] POST /identity/v3/users => generated 292 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:54:57.413805 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d7d1bf-7683-4a7c-96a2-440e8cea25bb admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:57.424843 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d7d1bf-7683-4a7c-96a2-440e8cea25bb admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:57.434544 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc918d7-a643-455b-829b-a77942bfcc39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:57.434544 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc918d7-a643-455b-829b-a77942bfcc39 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:57.434544 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc918d7-a643-455b-829b-a77942bfcc39 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:57.434544 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc918d7-a643-455b-829b-a77942bfcc39 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:57.434984 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc918d7-a643-455b-829b-a77942bfcc39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:57.434984 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc918d7-a643-455b-829b-a77942bfcc39 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:57.446395 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1064/2113] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:54:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:54:57.450869 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50c2054-fe24-41aa-90c1-67f0a6d4cd35 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:57.461735 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50c2054-fe24-41aa-90c1-67f0a6d4cd35 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:57.467832 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd93123-4cff-47a2-bcf4-2af5c8ca4d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:57.468733 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd93123-4cff-47a2-bcf4-2af5c8ca4d3f admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:57.469025 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd93123-4cff-47a2-bcf4-2af5c8ca4d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:57.469324 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd93123-4cff-47a2-bcf4-2af5c8ca4d3f admin admin] PATH_INFO: `/v3/projects/f06551316b854fcdb193b2ec592a644c/users/7a26f35b830941839f2e44e34d1902f1/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:57.483801 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:57.493887 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd93123-4cff-47a2-bcf4-2af5c8ca4d3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f06551316b854fcdb193b2ec592a644c, user_id=7a26f35b830941839f2e44e34d1902f1, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:57.496817 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd93123-4cff-47a2-bcf4-2af5c8ca4d3f admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:57.585861 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1050/2114] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:54:57 2022] PUT /identity/v3/projects/f06551316b854fcdb193b2ec592a644c/users/7a26f35b830941839f2e44e34d1902f1/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:57.587063 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc918d7-a643-455b-829b-a77942bfcc39 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:57.600168 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc918d7-a643-455b-829b-a77942bfcc39 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:57.666229 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44563e9-d5e7-4e7c-8954-d42fb9988e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:57.666882 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44563e9-d5e7-4e7c-8954-d42fb9988e51 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:57.667166 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44563e9-d5e7-4e7c-8954-d42fb9988e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:57.667527 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44563e9-d5e7-4e7c-8954-d42fb9988e51 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:57.668213 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44563e9-d5e7-4e7c-8954-d42fb9988e51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:57.670219 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44563e9-d5e7-4e7c-8954-d42fb9988e51 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:57.683600 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1065/2115] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:54:57 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:54:57.688303 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd93123-4cff-47a2-bcf4-2af5c8ca4d3f admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:57.703998 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd93123-4cff-47a2-bcf4-2af5c8ca4d3f admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:57.715930 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7541b37c-c158-4bde-9903-966e1d68b163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:57.717120 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7541b37c-c158-4bde-9903-966e1d68b163 admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:57.717428 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7541b37c-c158-4bde-9903-966e1d68b163 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:57.717843 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7541b37c-c158-4bde-9903-966e1d68b163 admin admin] PATH_INFO: `/v3/projects/f06551316b854fcdb193b2ec592a644c/users/7a26f35b830941839f2e44e34d1902f1/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:57.722120 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7541b37c-c158-4bde-9903-966e1d68b163 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f06551316b854fcdb193b2ec592a644c, user_id=7a26f35b830941839f2e44e34d1902f1, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:57.728910 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7541b37c-c158-4bde-9903-966e1d68b163 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:57.757402 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1051/2116] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:54:57 2022] PUT /identity/v3/projects/f06551316b854fcdb193b2ec592a644c/users/7a26f35b830941839f2e44e34d1902f1/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:57.890297 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44563e9-d5e7-4e7c-8954-d42fb9988e51 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:57.893349 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-c44563e9-d5e7-4e7c-8954-d42fb9988e51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:57.910757 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44563e9-d5e7-4e7c-8954-d42fb9988e51 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:58.032970 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a0ef0-d0c8-4c7f-bf07-35a180a868e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:58.034368 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a0ef0-d0c8-4c7f-bf07-35a180a868e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:58.034368 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a0ef0-d0c8-4c7f-bf07-35a180a868e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:58.034547 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a0ef0-d0c8-4c7f-bf07-35a180a868e3 admin admin] PATH_INFO: `/v3/users/137935860b364d698866aa5388668db8` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:58.036043 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4a0ef0-d0c8-4c7f-bf07-35a180a868e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=137935860b364d698866aa5388668db8)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:58.038212 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4a0ef0-d0c8-4c7f-bf07-35a180a868e3 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:58.287442 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-8e4a0ef0-d0c8-4c7f-bf07-35a180a868e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '137935860b364d698866aa5388668db8'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:58.300169 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-8e4a0ef0-d0c8-4c7f-bf07-35a180a868e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '137935860b364d698866aa5388668db8'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:58.304756 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-8e4a0ef0-d0c8-4c7f-bf07-35a180a868e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '137935860b364d698866aa5388668db8'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:58.330078 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-8e4a0ef0-d0c8-4c7f-bf07-35a180a868e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '137935860b364d698866aa5388668db8'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:58.508896 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-8e4a0ef0-d0c8-4c7f-bf07-35a180a868e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '137935860b364d698866aa5388668db8'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:58.520803 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1066/2117] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:54:57 2022] DELETE /identity/v3/users/137935860b364d698866aa5388668db8 => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:58.523209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc4dc6c-63bb-4743-88bc-43e56a59895f None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:58.523209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc4dc6c-63bb-4743-88bc-43e56a59895f None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:58.523739 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc4dc6c-63bb-4743-88bc-43e56a59895f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:58.553841 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-2bc4dc6c-63bb-4743-88bc-43e56a59895f None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:58.557435 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-2bc4dc6c-63bb-4743-88bc-43e56a59895f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:58.641973 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1052/2118] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:54:58.652318 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a0ef0-d0c8-4c7f-bf07-35a180a868e3 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:58.654018 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-8e4a0ef0-d0c8-4c7f-bf07-35a180a868e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:58.771890 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a0ef0-d0c8-4c7f-bf07-35a180a868e3 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:58.779517 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934bd2bb-6a1a-4a1d-bc86-2ce83ff2da62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:58.780368 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934bd2bb-6a1a-4a1d-bc86-2ce83ff2da62 service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:58.780829 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934bd2bb-6a1a-4a1d-bc86-2ce83ff2da62 service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:58.781255 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934bd2bb-6a1a-4a1d-bc86-2ce83ff2da62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:58.801989 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934bd2bb-6a1a-4a1d-bc86-2ce83ff2da62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:58.805408 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934bd2bb-6a1a-4a1d-bc86-2ce83ff2da62 service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:58.829486 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1067/2119] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:54:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:54:59.091157 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:59.194493 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:59.593793 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc4dc6c-63bb-4743-88bc-43e56a59895f None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:59.596184 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-2bc4dc6c-63bb-4743-88bc-43e56a59895f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:59.615917 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc4dc6c-63bb-4743-88bc-43e56a59895f None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:54:59.633951 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37a0bf-24af-4a4c-9372-02b39d905a08 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:59.635378 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37a0bf-24af-4a4c-9372-02b39d905a08 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:59.635875 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37a0bf-24af-4a4c-9372-02b39d905a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:59.652576 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e38c38-4ccd-402d-bdfa-e6ba74737100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:54:59.653830 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e38c38-4ccd-402d-bdfa-e6ba74737100 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:54:59.655953 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e38c38-4ccd-402d-bdfa-e6ba74737100 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:54:59.656516 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e38c38-4ccd-402d-bdfa-e6ba74737100 admin admin] PATH_INFO: `/v3/projects/1adf6d8fe0774070a02b77d1e712990b` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:54:59.659525 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e38c38-4ccd-402d-bdfa-e6ba74737100 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1adf6d8fe0774070a02b77d1e712990b)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:54:59.668395 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e38c38-4ccd-402d-bdfa-e6ba74737100 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:54:59.683728 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-be37a0bf-24af-4a4c-9372-02b39d905a08 None None] MFA Rules not processed for user `8c11dc2f4c764dc58458d9226efc878e`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:54:59.687820 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-be37a0bf-24af-4a4c-9372-02b39d905a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:54:59.790164 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1068/2120] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:54:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:54:59.844881 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-a0e38c38-4ccd-402d-bdfa-e6ba74737100 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1adf6d8fe0774070a02b77d1e712990b', 'request_id': 'req-a0e38c38-4ccd-402d-bdfa-e6ba74737100'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:59.852906 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-a0e38c38-4ccd-402d-bdfa-e6ba74737100 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1adf6d8fe0774070a02b77d1e712990b', 'request_id': 'req-a0e38c38-4ccd-402d-bdfa-e6ba74737100'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:59.922178 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:54:59.923178 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-a0e38c38-4ccd-402d-bdfa-e6ba74737100 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1adf6d8fe0774070a02b77d1e712990b', 'request_id': 'req-a0e38c38-4ccd-402d-bdfa-e6ba74737100'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:59.942703 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-a0e38c38-4ccd-402d-bdfa-e6ba74737100 admin admin] The token cache is being invalidate because project 1adf6d8fe0774070a02b77d1e712990b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:54:59.943387 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-a0e38c38-4ccd-402d-bdfa-e6ba74737100 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:59.944332 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-a0e38c38-4ccd-402d-bdfa-e6ba74737100 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:54:59.952424 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1053/2121] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:54:59 2022] DELETE /identity/v3/projects/1adf6d8fe0774070a02b77d1e712990b => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:54:59.958761 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be37a0bf-24af-4a4c-9372-02b39d905a08 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:54:59.985075 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be37a0bf-24af-4a4c-9372-02b39d905a08 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:00.083150 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb73a5-919c-4291-8fd8-1696fdf94b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:00.084665 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb73a5-919c-4291-8fd8-1696fdf94b73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:00.085054 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb73a5-919c-4291-8fd8-1696fdf94b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:00.085434 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb73a5-919c-4291-8fd8-1696fdf94b73 admin admin] PATH_INFO: `/v3/users/8bcc1e865e6549d4a545b5f78791ca2d` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:00.087542 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fb73a5-919c-4291-8fd8-1696fdf94b73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bcc1e865e6549d4a545b5f78791ca2d)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:00.090184 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fb73a5-919c-4291-8fd8-1696fdf94b73 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:00.375548 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-60fb73a5-919c-4291-8fd8-1696fdf94b73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bcc1e865e6549d4a545b5f78791ca2d'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:00.398260 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-60fb73a5-919c-4291-8fd8-1696fdf94b73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bcc1e865e6549d4a545b5f78791ca2d'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:00.404619 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-60fb73a5-919c-4291-8fd8-1696fdf94b73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bcc1e865e6549d4a545b5f78791ca2d'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:00.447213 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-60fb73a5-919c-4291-8fd8-1696fdf94b73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bcc1e865e6549d4a545b5f78791ca2d'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:00.539974 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-60fb73a5-919c-4291-8fd8-1696fdf94b73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bcc1e865e6549d4a545b5f78791ca2d'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:00.548845 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1069/2122] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:54:59 2022] DELETE /identity/v3/users/8bcc1e865e6549d4a545b5f78791ca2d => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:55:01.418102 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e38c38-4ccd-402d-bdfa-e6ba74737100 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:01.423780 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-a0e38c38-4ccd-402d-bdfa-e6ba74737100 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:01.449983 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e38c38-4ccd-402d-bdfa-e6ba74737100 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:01.579189 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31f62af-4fe8-4665-ba80-4041b4335e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:01.580610 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31f62af-4fe8-4665-ba80-4041b4335e8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:01.581676 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31f62af-4fe8-4665-ba80-4041b4335e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:01.581946 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31f62af-4fe8-4665-ba80-4041b4335e8e admin admin] PATH_INFO: `/v3/projects/08ca1a8b470d48d596408d9edd4c69b9` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:01.584468 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31f62af-4fe8-4665-ba80-4041b4335e8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=08ca1a8b470d48d596408d9edd4c69b9)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:01.587377 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31f62af-4fe8-4665-ba80-4041b4335e8e admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:01.749947 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-c31f62af-4fe8-4665-ba80-4041b4335e8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08ca1a8b470d48d596408d9edd4c69b9', 'request_id': 'req-c31f62af-4fe8-4665-ba80-4041b4335e8e'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:01.760278 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-c31f62af-4fe8-4665-ba80-4041b4335e8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08ca1a8b470d48d596408d9edd4c69b9', 'request_id': 'req-c31f62af-4fe8-4665-ba80-4041b4335e8e'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:01.832139 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:55:01.854618 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-c31f62af-4fe8-4665-ba80-4041b4335e8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08ca1a8b470d48d596408d9edd4c69b9', 'request_id': 'req-c31f62af-4fe8-4665-ba80-4041b4335e8e'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:01.870748 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-c31f62af-4fe8-4665-ba80-4041b4335e8e admin admin] The token cache is being invalidate because project 08ca1a8b470d48d596408d9edd4c69b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:55:01.871305 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-c31f62af-4fe8-4665-ba80-4041b4335e8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:01.872837 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-c31f62af-4fe8-4665-ba80-4041b4335e8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:01.875812 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1054/2123] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:55:01 2022] DELETE /identity/v3/projects/08ca1a8b470d48d596408d9edd4c69b9 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:55:01.880386 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb73a5-919c-4291-8fd8-1696fdf94b73 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:01.884154 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-60fb73a5-919c-4291-8fd8-1696fdf94b73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:01.906012 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb73a5-919c-4291-8fd8-1696fdf94b73 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:02.052187 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1fdffa-e13f-4252-8e43-b143b143209a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:02.053299 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1fdffa-e13f-4252-8e43-b143b143209a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:02.053582 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1fdffa-e13f-4252-8e43-b143b143209a admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:02.053855 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1fdffa-e13f-4252-8e43-b143b143209a admin admin] PATH_INFO: `/v3/users/b3c6ad69c2f6497aac901a9f16996024` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:02.055363 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1fdffa-e13f-4252-8e43-b143b143209a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3c6ad69c2f6497aac901a9f16996024)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:02.057466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1fdffa-e13f-4252-8e43-b143b143209a admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:02.333406 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-de1fdffa-e13f-4252-8e43-b143b143209a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3c6ad69c2f6497aac901a9f16996024'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:02.345443 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:55:02.365428 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-de1fdffa-e13f-4252-8e43-b143b143209a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3c6ad69c2f6497aac901a9f16996024'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:02.367028 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-de1fdffa-e13f-4252-8e43-b143b143209a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3c6ad69c2f6497aac901a9f16996024'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:02.387998 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-de1fdffa-e13f-4252-8e43-b143b143209a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3c6ad69c2f6497aac901a9f16996024'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:02.499938 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-de1fdffa-e13f-4252-8e43-b143b143209a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3c6ad69c2f6497aac901a9f16996024'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:02.515722 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1070/2124] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:55:01 2022] DELETE /identity/v3/users/b3c6ad69c2f6497aac901a9f16996024 => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:55:03.346695 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31f62af-4fe8-4665-ba80-4041b4335e8e admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:03.349553 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-c31f62af-4fe8-4665-ba80-4041b4335e8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:03.375110 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31f62af-4fe8-4665-ba80-4041b4335e8e admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:03.490707 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ecd3aa-bde4-430d-a660-8a8c165b5cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:03.493092 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ecd3aa-bde4-430d-a660-8a8c165b5cb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:03.508146 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ecd3aa-bde4-430d-a660-8a8c165b5cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:03.508389 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ecd3aa-bde4-430d-a660-8a8c165b5cb0 admin admin] PATH_INFO: `/v3/projects/4b8b773ffcab4a3cae7ef5232f763576` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:03.512836 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ecd3aa-bde4-430d-a660-8a8c165b5cb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b8b773ffcab4a3cae7ef5232f763576)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:03.517675 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ecd3aa-bde4-430d-a660-8a8c165b5cb0 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:03.656337 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:55:03.698047 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-68ecd3aa-bde4-430d-a660-8a8c165b5cb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b8b773ffcab4a3cae7ef5232f763576', 'request_id': 'req-68ecd3aa-bde4-430d-a660-8a8c165b5cb0'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:03.728789 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-68ecd3aa-bde4-430d-a660-8a8c165b5cb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b8b773ffcab4a3cae7ef5232f763576', 'request_id': 'req-68ecd3aa-bde4-430d-a660-8a8c165b5cb0'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:03.763936 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1fdffa-e13f-4252-8e43-b143b143209a admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:03.770810 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-de1fdffa-e13f-4252-8e43-b143b143209a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:03.792426 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1fdffa-e13f-4252-8e43-b143b143209a admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:03.827432 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-68ecd3aa-bde4-430d-a660-8a8c165b5cb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b8b773ffcab4a3cae7ef5232f763576', 'request_id': 'req-68ecd3aa-bde4-430d-a660-8a8c165b5cb0'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:03.850043 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-68ecd3aa-bde4-430d-a660-8a8c165b5cb0 admin admin] The token cache is being invalidate because project 4b8b773ffcab4a3cae7ef5232f763576 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:55:03.851158 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-68ecd3aa-bde4-430d-a660-8a8c165b5cb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:03.852134 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-68ecd3aa-bde4-430d-a660-8a8c165b5cb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:03.854257 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1055/2125] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:55:03 2022] DELETE /identity/v3/projects/4b8b773ffcab4a3cae7ef5232f763576 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:55:03.863647 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69be7cae-6c96-4d08-baec-247cabc0e1a4 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:03.864181 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69be7cae-6c96-4d08-baec-247cabc0e1a4 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:03.864628 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69be7cae-6c96-4d08-baec-247cabc0e1a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:03.908032 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-69be7cae-6c96-4d08-baec-247cabc0e1a4 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:55:03.909347 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-69be7cae-6c96-4d08-baec-247cabc0e1a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:03.981326 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3b76da-e262-4717-9efc-1796cbc654c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:03.982536 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3b76da-e262-4717-9efc-1796cbc654c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:03.983100 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3b76da-e262-4717-9efc-1796cbc654c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:03.983532 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3b76da-e262-4717-9efc-1796cbc654c7 admin admin] PATH_INFO: `/v3/users/87ea5ca395f142bfbea27f7e24c7e56c` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:03.985191 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3b76da-e262-4717-9efc-1796cbc654c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87ea5ca395f142bfbea27f7e24c7e56c)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:03.988556 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3b76da-e262-4717-9efc-1796cbc654c7 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:04.049991 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1056/2126] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:55:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:55:04.057817 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7b665-bc2d-4b8c-9b1a-009125ae3457 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:04.058116 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7b665-bc2d-4b8c-9b1a-009125ae3457 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:04.058585 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7b665-bc2d-4b8c-9b1a-009125ae3457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:04.115668 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-14e7b665-bc2d-4b8c-9b1a-009125ae3457 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:55:04.120411 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-14e7b665-bc2d-4b8c-9b1a-009125ae3457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:04.151384 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1057/2127] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:55:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:55:04.158289 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e7b665-bc2d-4b8c-9b1a-009125ae3457 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:04.204765 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e7b665-bc2d-4b8c-9b1a-009125ae3457 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:04.219115 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f215858-ca12-4149-958f-77d23e21d1d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:04.219797 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f215858-ca12-4149-958f-77d23e21d1d6 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:04.220082 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f215858-ca12-4149-958f-77d23e21d1d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:04.220356 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f215858-ca12-4149-958f-77d23e21d1d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:04.221092 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f215858-ca12-4149-958f-77d23e21d1d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:55:04.221451 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f215858-ca12-4149-958f-77d23e21d1d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:04.224534 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f215858-ca12-4149-958f-77d23e21d1d6 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:04.240796 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1058/2128] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 22:55:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:55:04.250993 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-fe3b76da-e262-4717-9efc-1796cbc654c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87ea5ca395f142bfbea27f7e24c7e56c'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:04.256357 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f215858-ca12-4149-958f-77d23e21d1d6 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:04.266162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-fe3b76da-e262-4717-9efc-1796cbc654c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87ea5ca395f142bfbea27f7e24c7e56c'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:04.268041 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-fe3b76da-e262-4717-9efc-1796cbc654c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87ea5ca395f142bfbea27f7e24c7e56c'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:04.278177 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f215858-ca12-4149-958f-77d23e21d1d6 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:04.284731 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ded4c1-801d-4c65-b89f-33417e35ae9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:04.285734 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ded4c1-801d-4c65-b89f-33417e35ae9c admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:04.286754 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ded4c1-801d-4c65-b89f-33417e35ae9c admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:04.286754 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ded4c1-801d-4c65-b89f-33417e35ae9c admin admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:04.287232 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ded4c1-801d-4c65-b89f-33417e35ae9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:04.290482 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ded4c1-801d-4c65-b89f-33417e35ae9c admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:04.294462 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-fe3b76da-e262-4717-9efc-1796cbc654c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87ea5ca395f142bfbea27f7e24c7e56c'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:04.458552 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-fe3b76da-e262-4717-9efc-1796cbc654c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87ea5ca395f142bfbea27f7e24c7e56c'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:04.467510 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1071/2129] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:55:03 2022] DELETE /identity/v3/users/87ea5ca395f142bfbea27f7e24c7e56c => generated 0 bytes in 704 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:55:04.472627 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464fb77e-74f2-4ad3-ac70-340e990ffe4d None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:04.473151 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464fb77e-74f2-4ad3-ac70-340e990ffe4d None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:04.473615 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464fb77e-74f2-4ad3-ac70-340e990ffe4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:04.485412 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1059/2130] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:55:04 2022] POST /identity/v3/projects => generated 381 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:55:04.486870 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ded4c1-801d-4c65-b89f-33417e35ae9c admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:04.489967 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-16ded4c1-801d-4c65-b89f-33417e35ae9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:04.508258 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-464fb77e-74f2-4ad3-ac70-340e990ffe4d None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:55:04.510175 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-464fb77e-74f2-4ad3-ac70-340e990ffe4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:04.557637 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:55:04.611278 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ded4c1-801d-4c65-b89f-33417e35ae9c admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:04.623841 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1161901-8ecb-49dd-9c00-418030f33bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:04.624531 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1161901-8ecb-49dd-9c00-418030f33bf5 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:04.624812 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1161901-8ecb-49dd-9c00-418030f33bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:04.625083 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1161901-8ecb-49dd-9c00-418030f33bf5 admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:04.625781 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1161901-8ecb-49dd-9c00-418030f33bf5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:04.627791 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1161901-8ecb-49dd-9c00-418030f33bf5 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:04.631104 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1072/2131] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:55:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:55:04.644161 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464fb77e-74f2-4ad3-ac70-340e990ffe4d None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:04.646575 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-464fb77e-74f2-4ad3-ac70-340e990ffe4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:04.760083 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1060/2132] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:55:04 2022] POST /identity/v3/users => generated 291 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:55:04.761272 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464fb77e-74f2-4ad3-ac70-340e990ffe4d None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:04.761709 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1161901-8ecb-49dd-9c00-418030f33bf5 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:04.780720 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd95158-8426-4de2-b1db-de70da3253f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:04.782555 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd95158-8426-4de2-b1db-de70da3253f8 service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:04.782555 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd95158-8426-4de2-b1db-de70da3253f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:04.782555 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd95158-8426-4de2-b1db-de70da3253f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:04.789651 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1161901-8ecb-49dd-9c00-418030f33bf5 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:04.805351 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b891cca0-8a06-4538-a5e8-c44b03d67a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:04.806017 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891cca0-8a06-4538-a5e8-c44b03d67a06 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:04.810244 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891cca0-8a06-4538-a5e8-c44b03d67a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:04.810244 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891cca0-8a06-4538-a5e8-c44b03d67a06 admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:04.810244 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b891cca0-8a06-4538-a5e8-c44b03d67a06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:04.810244 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b891cca0-8a06-4538-a5e8-c44b03d67a06 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:04.829817 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd95158-8426-4de2-b1db-de70da3253f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:04.830016 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1061/2133] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:55:04 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:55:04.831358 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b891cca0-8a06-4538-a5e8-c44b03d67a06 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:04.831783 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd95158-8426-4de2-b1db-de70da3253f8 service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:04.853818 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b891cca0-8a06-4538-a5e8-c44b03d67a06 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:04.871839 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def5b491-2eb4-40d6-9713-283924ddf51b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:04.872528 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def5b491-2eb4-40d6-9713-283924ddf51b admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:04.872811 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def5b491-2eb4-40d6-9713-283924ddf51b admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:04.873090 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def5b491-2eb4-40d6-9713-283924ddf51b admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:04.873832 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def5b491-2eb4-40d6-9713-283924ddf51b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:04.876813 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def5b491-2eb4-40d6-9713-283924ddf51b admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:04.881708 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1073/2134] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:55:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:55:04.904653 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1062/2135] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:55:04 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:55:04.909046 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd95158-8426-4de2-b1db-de70da3253f8 service neutron] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:04.923675 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd95158-8426-4de2-b1db-de70da3253f8 service neutron] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:04.936245 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3066d192-9c9d-41c6-bc96-abd089950212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:04.937219 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066d192-9c9d-41c6-bc96-abd089950212 admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:04.937510 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066d192-9c9d-41c6-bc96-abd089950212 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:04.937788 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066d192-9c9d-41c6-bc96-abd089950212 admin admin] PATH_INFO: `/v3/projects/2822a8cb96b94bf3bf525b0785c36934/users/5eadb243fd6a4517bca72e18dec30e7a/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:05.006407 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3066d192-9c9d-41c6-bc96-abd089950212 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2822a8cb96b94bf3bf525b0785c36934, user_id=5eadb243fd6a4517bca72e18dec30e7a, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:05.009671 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3066d192-9c9d-41c6-bc96-abd089950212 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:05.123775 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1074/2136] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:55:04 2022] PUT /identity/v3/projects/2822a8cb96b94bf3bf525b0785c36934/users/5eadb243fd6a4517bca72e18dec30e7a/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:55:05.132431 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7857c7d0-390e-4b02-a8af-10c336fc37fb None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:05.132756 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7857c7d0-390e-4b02-a8af-10c336fc37fb None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:05.133016 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7857c7d0-390e-4b02-a8af-10c336fc37fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:05.172831 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-7857c7d0-390e-4b02-a8af-10c336fc37fb None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:55:05.174074 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-7857c7d0-390e-4b02-a8af-10c336fc37fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:05.285564 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1063/2137] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:55:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:55:05.296471 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3066d192-9c9d-41c6-bc96-abd089950212 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:05.298660 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-3066d192-9c9d-41c6-bc96-abd089950212 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:05.434596 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3066d192-9c9d-41c6-bc96-abd089950212 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:05.440198 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5a3179-e008-4843-aa40-292167b60399 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:05.442468 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a3179-e008-4843-aa40-292167b60399 service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:05.442799 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a3179-e008-4843-aa40-292167b60399 service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:05.443109 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5a3179-e008-4843-aa40-292167b60399 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:05.459452 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5a3179-e008-4843-aa40-292167b60399 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:05.460501 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5a3179-e008-4843-aa40-292167b60399 service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:05.475262 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1075/2138] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:55:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:55:05.837001 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7857c7d0-390e-4b02-a8af-10c336fc37fb None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:05.838547 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-7857c7d0-390e-4b02-a8af-10c336fc37fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:05.852102 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7857c7d0-390e-4b02-a8af-10c336fc37fb None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:05.857668 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bebffc-0711-489e-8cf6-f850d1a8cd12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:05.858736 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bebffc-0711-489e-8cf6-f850d1a8cd12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:05.859042 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bebffc-0711-489e-8cf6-f850d1a8cd12 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:05.859487 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bebffc-0711-489e-8cf6-f850d1a8cd12 admin admin] PATH_INFO: `/v3/users/51ee6ca4f6084e948e0ed3e5f4ca26ec` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:05.860838 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bebffc-0711-489e-8cf6-f850d1a8cd12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51ee6ca4f6084e948e0ed3e5f4ca26ec)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:05.862965 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bebffc-0711-489e-8cf6-f850d1a8cd12 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:05.919284 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5a3179-e008-4843-aa40-292167b60399 service neutron] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:05.923640 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-3e5a3179-e008-4843-aa40-292167b60399 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:05.939166 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5a3179-e008-4843-aa40-292167b60399 service neutron] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:05.945981 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b258b4fa-0a8c-46eb-98af-14dba00dc30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:05.947197 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b258b4fa-0a8c-46eb-98af-14dba00dc30f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:05.947622 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b258b4fa-0a8c-46eb-98af-14dba00dc30f admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:05.948140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b258b4fa-0a8c-46eb-98af-14dba00dc30f admin admin] PATH_INFO: `/v3/projects/40d62acc2a1a40aaa2a5cb7b4ef9072b` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:05.949932 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b258b4fa-0a8c-46eb-98af-14dba00dc30f admin admin] RBAC: Authorizing `identity:delete_project(project_id=40d62acc2a1a40aaa2a5cb7b4ef9072b)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:05.952661 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b258b4fa-0a8c-46eb-98af-14dba00dc30f admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:06.224924 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-40bebffc-0711-489e-8cf6-f850d1a8cd12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51ee6ca4f6084e948e0ed3e5f4ca26ec'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:06.242112 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-b258b4fa-0a8c-46eb-98af-14dba00dc30f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40d62acc2a1a40aaa2a5cb7b4ef9072b', 'request_id': 'req-b258b4fa-0a8c-46eb-98af-14dba00dc30f'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:06.252926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-40bebffc-0711-489e-8cf6-f850d1a8cd12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51ee6ca4f6084e948e0ed3e5f4ca26ec'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:06.255357 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-40bebffc-0711-489e-8cf6-f850d1a8cd12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51ee6ca4f6084e948e0ed3e5f4ca26ec'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:06.259929 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-b258b4fa-0a8c-46eb-98af-14dba00dc30f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40d62acc2a1a40aaa2a5cb7b4ef9072b', 'request_id': 'req-b258b4fa-0a8c-46eb-98af-14dba00dc30f'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:06.275709 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-40bebffc-0711-489e-8cf6-f850d1a8cd12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51ee6ca4f6084e948e0ed3e5f4ca26ec'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:06.276982 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-40bebffc-0711-489e-8cf6-f850d1a8cd12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51ee6ca4f6084e948e0ed3e5f4ca26ec'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:06.370692 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-b258b4fa-0a8c-46eb-98af-14dba00dc30f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40d62acc2a1a40aaa2a5cb7b4ef9072b', 'request_id': 'req-b258b4fa-0a8c-46eb-98af-14dba00dc30f'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:06.383660 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-b258b4fa-0a8c-46eb-98af-14dba00dc30f admin admin] The token cache is being invalidate because project 40d62acc2a1a40aaa2a5cb7b4ef9072b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:55:06.384011 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-b258b4fa-0a8c-46eb-98af-14dba00dc30f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:06.387377 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-b258b4fa-0a8c-46eb-98af-14dba00dc30f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:06.388544 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1076/2139] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:55:05 2022] DELETE /identity/v3/projects/40d62acc2a1a40aaa2a5cb7b4ef9072b => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:55:06.393457 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76452e50-17db-4a21-b391-80f692119dff None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:06.393940 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76452e50-17db-4a21-b391-80f692119dff None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:06.394225 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76452e50-17db-4a21-b391-80f692119dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:06.415029 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-76452e50-17db-4a21-b391-80f692119dff None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:55:06.416279 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-76452e50-17db-4a21-b391-80f692119dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:06.504160 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1077/2140] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:55:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:55:06.509292 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422348e1-5c09-4004-92fe-a1a432e51a5f None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:06.509292 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422348e1-5c09-4004-92fe-a1a432e51a5f None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:06.509292 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422348e1-5c09-4004-92fe-a1a432e51a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:06.531054 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-422348e1-5c09-4004-92fe-a1a432e51a5f None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:55:06.532769 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-422348e1-5c09-4004-92fe-a1a432e51a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:06.537133 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1064/2141] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:55:05 2022] DELETE /identity/v3/users/51ee6ca4f6084e948e0ed3e5f4ca26ec => generated 0 bytes in 701 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:55:06.541831 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c7087c-0429-41c4-9e5e-7c7571ffacc1 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:06.543201 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c7087c-0429-41c4-9e5e-7c7571ffacc1 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:06.543201 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c7087c-0429-41c4-9e5e-7c7571ffacc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:06.559964 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1078/2142] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:55:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:55:06.564147 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422348e1-5c09-4004-92fe-a1a432e51a5f None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:06.576841 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422348e1-5c09-4004-92fe-a1a432e51a5f None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:06.612840 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-b0c7087c-0429-41c4-9e5e-7c7571ffacc1 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:55:06.615922 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-b0c7087c-0429-41c4-9e5e-7c7571ffacc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:06.658533 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cab88e-68a0-456d-a1a9-b99421a06adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:06.659348 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cab88e-68a0-456d-a1a9-b99421a06adb admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:06.659641 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cab88e-68a0-456d-a1a9-b99421a06adb admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:06.659904 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cab88e-68a0-456d-a1a9-b99421a06adb admin admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:06.660665 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cab88e-68a0-456d-a1a9-b99421a06adb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:55:06.660978 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cab88e-68a0-456d-a1a9-b99421a06adb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:06.668544 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cab88e-68a0-456d-a1a9-b99421a06adb admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:06.720402 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1079/2143] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 22:55:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:55:06.728141 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cab88e-68a0-456d-a1a9-b99421a06adb admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:06.741323 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1065/2144] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:55:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:55:06.747278 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cab88e-68a0-456d-a1a9-b99421a06adb admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:06.752834 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce19eda-4043-491c-8584-97faf35cb51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:06.753516 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce19eda-4043-491c-8584-97faf35cb51f admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:06.754333 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce19eda-4043-491c-8584-97faf35cb51f admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:06.754617 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce19eda-4043-491c-8584-97faf35cb51f admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:06.755373 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce19eda-4043-491c-8584-97faf35cb51f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:06.757363 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce19eda-4043-491c-8584-97faf35cb51f admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:06.764090 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c7087c-0429-41c4-9e5e-7c7571ffacc1 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:06.765928 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-b0c7087c-0429-41c4-9e5e-7c7571ffacc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:06.851395 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c7087c-0429-41c4-9e5e-7c7571ffacc1 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:06.856008 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba6e3da-05bf-4919-876f-41aedc77ea35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:06.856676 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6e3da-05bf-4919-876f-41aedc77ea35 service neutron] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:06.856964 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6e3da-05bf-4919-876f-41aedc77ea35 service neutron] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:06.861251 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6e3da-05bf-4919-876f-41aedc77ea35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:06.867987 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba6e3da-05bf-4919-876f-41aedc77ea35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:06.869494 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba6e3da-05bf-4919-876f-41aedc77ea35 service neutron] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:06.881030 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1066/2145] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:55:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:55:06.882431 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a5f0b3-af39-4cd7-a374-f59138fecd7c None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:06.882733 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a5f0b3-af39-4cd7-a374-f59138fecd7c None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:06.883033 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a5f0b3-af39-4cd7-a374-f59138fecd7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:06.897769 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1080/2146] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:55:06 2022] POST /identity/v3/projects => generated 373 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:55:06.898824 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce19eda-4043-491c-8584-97faf35cb51f admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:06.913870 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:55:06.934993 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-77a5f0b3-af39-4cd7-a374-f59138fecd7c None None] MFA Rules not processed for user `7a26f35b830941839f2e44e34d1902f1`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:55:06.937062 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-77a5f0b3-af39-4cd7-a374-f59138fecd7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:07.003556 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce19eda-4043-491c-8584-97faf35cb51f admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:07.014015 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1122700b-ee63-47fa-9d74-cc679aa37c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:07.014922 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1122700b-ee63-47fa-9d74-cc679aa37c02 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:07.015341 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1122700b-ee63-47fa-9d74-cc679aa37c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:07.016053 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1122700b-ee63-47fa-9d74-cc679aa37c02 admin admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:07.016974 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1122700b-ee63-47fa-9d74-cc679aa37c02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:07.020174 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1122700b-ee63-47fa-9d74-cc679aa37c02 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:07.116618 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1067/2147] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:55:06 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:55:07.131932 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a5f0b3-af39-4cd7-a374-f59138fecd7c None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:07.133182 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1081/2148] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:55:06 2022] POST /identity/v3/users => generated 287 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:55:07.134400 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-77a5f0b3-af39-4cd7-a374-f59138fecd7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:07.138242 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1122700b-ee63-47fa-9d74-cc679aa37c02 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:07.154140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1122700b-ee63-47fa-9d74-cc679aa37c02 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:07.161397 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddb2ddb-acdb-4066-854f-d0a8d519d144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:07.162605 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb2ddb-acdb-4066-854f-d0a8d519d144 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:07.164197 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb2ddb-acdb-4066-854f-d0a8d519d144 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:07.164706 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb2ddb-acdb-4066-854f-d0a8d519d144 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:07.166274 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddb2ddb-acdb-4066-854f-d0a8d519d144 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:07.169402 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddb2ddb-acdb-4066-854f-d0a8d519d144 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:07.187602 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1082/2149] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:55:07 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:55:07.192195 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddb2ddb-acdb-4066-854f-d0a8d519d144 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:07.206319 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddb2ddb-acdb-4066-854f-d0a8d519d144 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:07.215188 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dbede0-1ba0-4af4-8cac-018ffacd6482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:07.216079 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dbede0-1ba0-4af4-8cac-018ffacd6482 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:07.216690 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dbede0-1ba0-4af4-8cac-018ffacd6482 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:07.217036 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dbede0-1ba0-4af4-8cac-018ffacd6482 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:07.218177 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dbede0-1ba0-4af4-8cac-018ffacd6482 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:07.219964 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dbede0-1ba0-4af4-8cac-018ffacd6482 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:07.242918 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1083/2150] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:55:07 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:55:07.246529 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dbede0-1ba0-4af4-8cac-018ffacd6482 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:07.265913 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dbede0-1ba0-4af4-8cac-018ffacd6482 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:07.284106 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9a2b68-c20f-4f6e-a847-a6b893ac52e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:07.285675 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a2b68-c20f-4f6e-a847-a6b893ac52e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:07.286059 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a2b68-c20f-4f6e-a847-a6b893ac52e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:07.286436 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a2b68-c20f-4f6e-a847-a6b893ac52e1 admin admin] PATH_INFO: `/v3/projects/ecc2e51ff1354580ae28f9f03f9c916c/users/ae84a77e67764396bd2a7564f64df147/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:07.307792 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a5f0b3-af39-4cd7-a374-f59138fecd7c None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:07.322763 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9a2b68-c20f-4f6e-a847-a6b893ac52e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecc2e51ff1354580ae28f9f03f9c916c, user_id=ae84a77e67764396bd2a7564f64df147, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:07.323125 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e31f96-7c79-4697-9f88-2d93a4d7648c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bfdb655a2a44b33ab724daaf73a294b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:07.323687 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e31f96-7c79-4697-9f88-2d93a4d7648c service nova] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:07.326383 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e31f96-7c79-4697-9f88-2d93a4d7648c service nova] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:07.326383 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e31f96-7c79-4697-9f88-2d93a4d7648c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:07.326961 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-d6e31f96-7c79-4697-9f88-2d93a4d7648c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:07.328384 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9a2b68-c20f-4f6e-a847-a6b893ac52e1 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:07.430335 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1084/2151] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:55:07 2022] PUT /identity/v3/projects/ecc2e51ff1354580ae28f9f03f9c916c/users/ae84a77e67764396bd2a7564f64df147/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:55:07.434097 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:55:07.438361 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e31f96-7c79-4697-9f88-2d93a4d7648c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:07.440864 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba45ef-bce5-4704-bab2-3ee3faa188b5 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:07.441268 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba45ef-bce5-4704-bab2-3ee3faa188b5 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:07.441655 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba45ef-bce5-4704-bab2-3ee3faa188b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:07.452891 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e31f96-7c79-4697-9f88-2d93a4d7648c service nova] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:07.501468 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1068/2152] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:55:07 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:55:07.536352 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-06ba45ef-bce5-4704-bab2-3ee3faa188b5 None None] MFA Rules not processed for user `ae84a77e67764396bd2a7564f64df147`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:55:07.537872 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-06ba45ef-bce5-4704-bab2-3ee3faa188b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:07.671310 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1085/2153] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:55:07 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:55:07.683057 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e31f96-7c79-4697-9f88-2d93a4d7648c service nova] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:07.684442 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-d6e31f96-7c79-4697-9f88-2d93a4d7648c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:07.758335 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e31f96-7c79-4697-9f88-2d93a4d7648c service nova] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:07.840756 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789bdc87-e907-45cf-b493-14257e84681f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:07.841835 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789bdc87-e907-45cf-b493-14257e84681f service cinder] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:07.842220 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789bdc87-e907-45cf-b493-14257e84681f service cinder] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:07.842894 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789bdc87-e907-45cf-b493-14257e84681f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:07.888213 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789bdc87-e907-45cf-b493-14257e84681f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:07.890704 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789bdc87-e907-45cf-b493-14257e84681f service cinder] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:07.920525 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1069/2154] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:55:07 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:55:07.937238 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1963c278-c0cd-4843-adb4-d7ee618be833 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:07.937454 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1963c278-c0cd-4843-adb4-d7ee618be833 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:07.937764 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1963c278-c0cd-4843-adb4-d7ee618be833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:08.023495 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-1963c278-c0cd-4843-adb4-d7ee618be833 None None] MFA Rules not processed for user `5eadb243fd6a4517bca72e18dec30e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:55:08.026258 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-1963c278-c0cd-4843-adb4-d7ee618be833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:08.153908 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1086/2155] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:55:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:55:08.167144 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789bdc87-e907-45cf-b493-14257e84681f service cinder] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:08.179597 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789bdc87-e907-45cf-b493-14257e84681f service cinder] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:08.188386 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cf173e-41a0-4071-95d8-c537ad609faf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:08.190908 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf173e-41a0-4071-95d8-c537ad609faf service cinder] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:08.190908 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf173e-41a0-4071-95d8-c537ad609faf service cinder] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:08.193890 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf173e-41a0-4071-95d8-c537ad609faf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:08.231221 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cf173e-41a0-4071-95d8-c537ad609faf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:08.233770 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cf173e-41a0-4071-95d8-c537ad609faf service cinder] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:08.259069 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1070/2156] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:55:08 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:55:08.512567 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1963c278-c0cd-4843-adb4-d7ee618be833 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:08.513998 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-1963c278-c0cd-4843-adb4-d7ee618be833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:08.605237 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1963c278-c0cd-4843-adb4-d7ee618be833 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:08.609907 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d0715a-66b0-44a8-ab73-92368123dc1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:08.610734 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d0715a-66b0-44a8-ab73-92368123dc1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:08.614510 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d0715a-66b0-44a8-ab73-92368123dc1c admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:08.614510 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d0715a-66b0-44a8-ab73-92368123dc1c admin admin] PATH_INFO: `/v3/projects/bd00b813323b4a429b455e1b34360042` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:08.614510 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d0715a-66b0-44a8-ab73-92368123dc1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd00b813323b4a429b455e1b34360042)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:08.615089 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d0715a-66b0-44a8-ab73-92368123dc1c admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:08.739157 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-b8d0715a-66b0-44a8-ab73-92368123dc1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd00b813323b4a429b455e1b34360042', 'request_id': 'req-b8d0715a-66b0-44a8-ab73-92368123dc1c'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:08.747456 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-b8d0715a-66b0-44a8-ab73-92368123dc1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd00b813323b4a429b455e1b34360042', 'request_id': 'req-b8d0715a-66b0-44a8-ab73-92368123dc1c'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:08.888186 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-b8d0715a-66b0-44a8-ab73-92368123dc1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd00b813323b4a429b455e1b34360042', 'request_id': 'req-b8d0715a-66b0-44a8-ab73-92368123dc1c'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:08.905515 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-b8d0715a-66b0-44a8-ab73-92368123dc1c admin admin] The token cache is being invalidate because project bd00b813323b4a429b455e1b34360042 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:55:08.905845 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-b8d0715a-66b0-44a8-ab73-92368123dc1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:08.906474 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-b8d0715a-66b0-44a8-ab73-92368123dc1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:08.909179 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1087/2157] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:55:08 2022] DELETE /identity/v3/projects/bd00b813323b4a429b455e1b34360042 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:55:08.913142 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cf173e-41a0-4071-95d8-c537ad609faf service cinder] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:08.929226 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cf173e-41a0-4071-95d8-c537ad609faf service cinder] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:09.007606 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fb3703-9415-4a1c-b8ed-4d3d603dd093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:09.007606 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb3703-9415-4a1c-b8ed-4d3d603dd093 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:09.007606 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb3703-9415-4a1c-b8ed-4d3d603dd093 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:09.008056 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb3703-9415-4a1c-b8ed-4d3d603dd093 admin admin] PATH_INFO: `/v3/users/13a963066e3d4a208d3821f0114921c6` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:09.009762 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fb3703-9415-4a1c-b8ed-4d3d603dd093 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13a963066e3d4a208d3821f0114921c6)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:09.012307 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fb3703-9415-4a1c-b8ed-4d3d603dd093 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:09.216079 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:55:09.262328 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-a2fb3703-9415-4a1c-b8ed-4d3d603dd093 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13a963066e3d4a208d3821f0114921c6'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:09.273299 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-a2fb3703-9415-4a1c-b8ed-4d3d603dd093 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13a963066e3d4a208d3821f0114921c6'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:09.274256 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-a2fb3703-9415-4a1c-b8ed-4d3d603dd093 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13a963066e3d4a208d3821f0114921c6'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:09.296056 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-a2fb3703-9415-4a1c-b8ed-4d3d603dd093 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13a963066e3d4a208d3821f0114921c6'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:09.297091 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-a2fb3703-9415-4a1c-b8ed-4d3d603dd093 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13a963066e3d4a208d3821f0114921c6'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:09.433856 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1071/2158] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:55:08 2022] DELETE /identity/v3/users/13a963066e3d4a208d3821f0114921c6 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:55:09.582749 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:55:10.148737 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d0715a-66b0-44a8-ab73-92368123dc1c admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:10.150384 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-b8d0715a-66b0-44a8-ab73-92368123dc1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:10.172965 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d0715a-66b0-44a8-ab73-92368123dc1c admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:10.246191 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eef5a1c-5cb0-4e15-ad03-015a3ed8f002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:10.247368 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eef5a1c-5cb0-4e15-ad03-015a3ed8f002 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:10.248175 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eef5a1c-5cb0-4e15-ad03-015a3ed8f002 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:10.248577 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eef5a1c-5cb0-4e15-ad03-015a3ed8f002 admin admin] PATH_INFO: `/v3/projects/bbafea3dad654f12b4921efb1991474a` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:10.251659 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eef5a1c-5cb0-4e15-ad03-015a3ed8f002 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbafea3dad654f12b4921efb1991474a)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:10.254606 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eef5a1c-5cb0-4e15-ad03-015a3ed8f002 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:10.317119 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdcd205-17c4-497d-ae1d-553c1b9adaff None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:10.317598 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdcd205-17c4-497d-ae1d-553c1b9adaff None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:10.317866 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdcd205-17c4-497d-ae1d-553c1b9adaff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:10.353371 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-7cdcd205-17c4-497d-ae1d-553c1b9adaff None None] MFA Rules not processed for user `5eadb243fd6a4517bca72e18dec30e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:55:10.355887 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-7cdcd205-17c4-497d-ae1d-553c1b9adaff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:10.424805 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1072/2159] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:55:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:55:10.448216 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-6eef5a1c-5cb0-4e15-ad03-015a3ed8f002 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbafea3dad654f12b4921efb1991474a', 'request_id': 'req-6eef5a1c-5cb0-4e15-ad03-015a3ed8f002'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:10.454964 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-6eef5a1c-5cb0-4e15-ad03-015a3ed8f002 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbafea3dad654f12b4921efb1991474a', 'request_id': 'req-6eef5a1c-5cb0-4e15-ad03-015a3ed8f002'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:10.577924 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-6eef5a1c-5cb0-4e15-ad03-015a3ed8f002 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbafea3dad654f12b4921efb1991474a', 'request_id': 'req-6eef5a1c-5cb0-4e15-ad03-015a3ed8f002'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:10.605612 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-6eef5a1c-5cb0-4e15-ad03-015a3ed8f002 admin admin] The token cache is being invalidate because project bbafea3dad654f12b4921efb1991474a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:55:10.605976 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-6eef5a1c-5cb0-4e15-ad03-015a3ed8f002 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:10.606596 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-6eef5a1c-5cb0-4e15-ad03-015a3ed8f002 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:10.608607 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1088/2160] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:55:10 2022] DELETE /identity/v3/projects/bbafea3dad654f12b4921efb1991474a => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:55:10.613691 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bfbc92-ec24-4756-856f-1281b61338d9 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:10.614241 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bfbc92-ec24-4756-856f-1281b61338d9 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:10.614670 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bfbc92-ec24-4756-856f-1281b61338d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:10.647688 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-07bfbc92-ec24-4756-856f-1281b61338d9 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:55:10.647688 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-07bfbc92-ec24-4756-856f-1281b61338d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:10.729046 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1073/2161] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:55:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:55:10.737107 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc732e9-9cf3-4a1e-b787-343e63dd92df None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:10.737401 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc732e9-9cf3-4a1e-b787-343e63dd92df None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:10.737686 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc732e9-9cf3-4a1e-b787-343e63dd92df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:10.781710 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-1fc732e9-9cf3-4a1e-b787-343e63dd92df None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:55:10.783312 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-1fc732e9-9cf3-4a1e-b787-343e63dd92df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:10.807418 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1089/2162] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:55:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:55:10.811777 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bfbc92-ec24-4756-856f-1281b61338d9 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:10.822725 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bfbc92-ec24-4756-856f-1281b61338d9 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:10.828216 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af37ce-1c14-4427-810c-844b4f51b6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:10.828941 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af37ce-1c14-4427-810c-844b4f51b6ad admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:10.829223 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af37ce-1c14-4427-810c-844b4f51b6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:10.829499 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af37ce-1c14-4427-810c-844b4f51b6ad admin admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:10.830403 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03af37ce-1c14-4427-810c-844b4f51b6ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:55:10.830761 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03af37ce-1c14-4427-810c-844b4f51b6ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:10.833041 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03af37ce-1c14-4427-810c-844b4f51b6ad admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:10.854418 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc732e9-9cf3-4a1e-b787-343e63dd92df None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:10.862426 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1074/2163] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 22:55:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:55:10.866752 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc732e9-9cf3-4a1e-b787-343e63dd92df None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:10.876009 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645e1a43-cfc7-4c64-91c7-e1a6f70de3a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:10.876685 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e1a43-cfc7-4c64-91c7-e1a6f70de3a0 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:10.876966 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e1a43-cfc7-4c64-91c7-e1a6f70de3a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:10.877240 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e1a43-cfc7-4c64-91c7-e1a6f70de3a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:10.877937 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645e1a43-cfc7-4c64-91c7-e1a6f70de3a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:10.879889 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645e1a43-cfc7-4c64-91c7-e1a6f70de3a0 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:11.079735 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1090/2164] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:55:10 2022] POST /identity/v3/projects => generated 377 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:55:11.082927 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af37ce-1c14-4427-810c-844b4f51b6ad admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:11.093980 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af37ce-1c14-4427-810c-844b4f51b6ad admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:11.166604 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb43afe-2905-4251-9f30-06a3bcc1f4a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:11.167646 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb43afe-2905-4251-9f30-06a3bcc1f4a6 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:11.168123 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb43afe-2905-4251-9f30-06a3bcc1f4a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:11.168537 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb43afe-2905-4251-9f30-06a3bcc1f4a6 admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:11.169517 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb43afe-2905-4251-9f30-06a3bcc1f4a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:11.172314 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb43afe-2905-4251-9f30-06a3bcc1f4a6 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:11.177185 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:55:11.311155 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1075/2165] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:55:11 2022] POST /identity/v3/users => generated 289 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:55:11.315257 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645e1a43-cfc7-4c64-91c7-e1a6f70de3a0 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:11.327115 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645e1a43-cfc7-4c64-91c7-e1a6f70de3a0 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:11.339542 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59744429-d55b-4e7d-98cd-86028b568f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:11.340520 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59744429-d55b-4e7d-98cd-86028b568f37 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:11.340981 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59744429-d55b-4e7d-98cd-86028b568f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:11.341728 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59744429-d55b-4e7d-98cd-86028b568f37 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:11.342639 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59744429-d55b-4e7d-98cd-86028b568f37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:11.344471 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59744429-d55b-4e7d-98cd-86028b568f37 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:11.372906 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1091/2166] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:55:11 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:55:11.377304 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb43afe-2905-4251-9f30-06a3bcc1f4a6 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:11.389470 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb43afe-2905-4251-9f30-06a3bcc1f4a6 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:11.395189 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b893509-c485-4331-82c7-d064effba9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:11.395969 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b893509-c485-4331-82c7-d064effba9a8 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:11.401988 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b893509-c485-4331-82c7-d064effba9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:11.401988 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b893509-c485-4331-82c7-d064effba9a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:11.401988 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b893509-c485-4331-82c7-d064effba9a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:11.401988 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b893509-c485-4331-82c7-d064effba9a8 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:11.410804 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1076/2167] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:55:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:55:11.416835 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59744429-d55b-4e7d-98cd-86028b568f37 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:11.429840 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59744429-d55b-4e7d-98cd-86028b568f37 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:11.434854 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d69807-d11f-4c8b-b5f5-fe7df91629d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:11.435842 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d69807-d11f-4c8b-b5f5-fe7df91629d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:11.436224 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d69807-d11f-4c8b-b5f5-fe7df91629d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:11.436573 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d69807-d11f-4c8b-b5f5-fe7df91629d9 admin admin] PATH_INFO: `/v3/projects/a4d9dacbc48343d38b6a31ff3f2d34a1/users/b183096b8bdd4f8891612fe72b15c5d1/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:11.453851 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d69807-d11f-4c8b-b5f5-fe7df91629d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4d9dacbc48343d38b6a31ff3f2d34a1, user_id=b183096b8bdd4f8891612fe72b15c5d1, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:11.463914 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d69807-d11f-4c8b-b5f5-fe7df91629d9 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:11.513003 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:55:11.586189 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1092/2168] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:55:11 2022] PUT /identity/v3/projects/a4d9dacbc48343d38b6a31ff3f2d34a1/users/b183096b8bdd4f8891612fe72b15c5d1/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:55:11.592014 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb7305-ac06-40be-8ffd-c98eadf8e2f5 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:11.592298 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb7305-ac06-40be-8ffd-c98eadf8e2f5 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:11.592777 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb7305-ac06-40be-8ffd-c98eadf8e2f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:11.621494 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60ebe7-3c63-49de-b7d2-f857ac621391 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:11.621795 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60ebe7-3c63-49de-b7d2-f857ac621391 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:11.622083 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60ebe7-3c63-49de-b7d2-f857ac621391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:11.639893 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-8efb7305-ac06-40be-8ffd-c98eadf8e2f5 None None] MFA Rules not processed for user `b183096b8bdd4f8891612fe72b15c5d1`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:55:11.641726 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-8efb7305-ac06-40be-8ffd-c98eadf8e2f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:11.657490 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-5f60ebe7-3c63-49de-b7d2-f857ac621391 None None] MFA Rules not processed for user `a0a765202a9645a1acc332f559537458`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:55:11.658897 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-5f60ebe7-3c63-49de-b7d2-f857ac621391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:11.769179 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1077/2169] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:55:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:55:11.774541 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efb7305-ac06-40be-8ffd-c98eadf8e2f5 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:11.794369 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1093/2170] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:55:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:55:11.881066 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efb7305-ac06-40be-8ffd-c98eadf8e2f5 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:11.890613 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5399ea0a-eec6-4427-a1b8-d844819598cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:11.891428 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5399ea0a-eec6-4427-a1b8-d844819598cf admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:11.891722 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5399ea0a-eec6-4427-a1b8-d844819598cf admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:11.892257 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5399ea0a-eec6-4427-a1b8-d844819598cf admin admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:11.893136 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5399ea0a-eec6-4427-a1b8-d844819598cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:11.896325 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5399ea0a-eec6-4427-a1b8-d844819598cf admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:12.026826 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1078/2171] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:55:11 2022] POST /identity/v3/projects => generated 377 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:55:12.030284 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f60ebe7-3c63-49de-b7d2-f857ac621391 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:12.048386 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f60ebe7-3c63-49de-b7d2-f857ac621391 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:12.130935 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdfe2f3-c5a9-407b-9468-2a542b8a8699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:12.131739 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdfe2f3-c5a9-407b-9468-2a542b8a8699 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:12.132123 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdfe2f3-c5a9-407b-9468-2a542b8a8699 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:12.133089 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdfe2f3-c5a9-407b-9468-2a542b8a8699 admin admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:12.133983 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdfe2f3-c5a9-407b-9468-2a542b8a8699 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:12.144931 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdfe2f3-c5a9-407b-9468-2a542b8a8699 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:12.308435 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1094/2172] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:55:12 2022] POST /identity/v3/users => generated 295 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:55:12.312394 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5399ea0a-eec6-4427-a1b8-d844819598cf admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:12.324866 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5399ea0a-eec6-4427-a1b8-d844819598cf admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:12.329589 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a485b7-acb1-48b3-b190-686ff93ffdf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:12.330381 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a485b7-acb1-48b3-b190-686ff93ffdf6 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:12.330605 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a485b7-acb1-48b3-b190-686ff93ffdf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:12.331018 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a485b7-acb1-48b3-b190-686ff93ffdf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:12.332032 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a485b7-acb1-48b3-b190-686ff93ffdf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:12.336507 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a485b7-acb1-48b3-b190-686ff93ffdf6 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:12.348417 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1079/2173] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:55:12 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:55:12.352994 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdfe2f3-c5a9-407b-9468-2a542b8a8699 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:12.364145 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdfe2f3-c5a9-407b-9468-2a542b8a8699 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:12.373757 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece68a98-535e-46e9-be78-e2a61680d649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:12.375321 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece68a98-535e-46e9-be78-e2a61680d649 admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:12.375321 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece68a98-535e-46e9-be78-e2a61680d649 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:12.375811 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece68a98-535e-46e9-be78-e2a61680d649 admin admin] PATH_INFO: `/v3/projects/ed4ec5cacb8042149ecd457e00f549e0/users/2201178187464803a557d9530be7e0ba/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:12.393797 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece68a98-535e-46e9-be78-e2a61680d649 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed4ec5cacb8042149ecd457e00f549e0, user_id=2201178187464803a557d9530be7e0ba, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:12.396603 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece68a98-535e-46e9-be78-e2a61680d649 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:12.481050 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1095/2174] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:55:12 2022] PUT /identity/v3/projects/ed4ec5cacb8042149ecd457e00f549e0/users/2201178187464803a557d9530be7e0ba/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:55:12.482749 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a485b7-acb1-48b3-b190-686ff93ffdf6 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:12.495875 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a485b7-acb1-48b3-b190-686ff93ffdf6 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:12.567965 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38515003-2a75-4af3-9186-04afded65dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:12.567965 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38515003-2a75-4af3-9186-04afded65dc0 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:12.567965 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38515003-2a75-4af3-9186-04afded65dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:12.567965 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38515003-2a75-4af3-9186-04afded65dc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:12.568340 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38515003-2a75-4af3-9186-04afded65dc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:12.568688 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38515003-2a75-4af3-9186-04afded65dc0 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:12.579240 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1080/2175] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:55:12 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:55:12.583560 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece68a98-535e-46e9-be78-e2a61680d649 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:12.596155 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece68a98-535e-46e9-be78-e2a61680d649 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:12.601794 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a875c6-2779-4381-9a92-2ed2a49f8ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:12.602825 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a875c6-2779-4381-9a92-2ed2a49f8ebc admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:12.603095 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a875c6-2779-4381-9a92-2ed2a49f8ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:12.603374 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a875c6-2779-4381-9a92-2ed2a49f8ebc admin admin] PATH_INFO: `/v3/projects/ed4ec5cacb8042149ecd457e00f549e0/users/2201178187464803a557d9530be7e0ba/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:12.605503 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a875c6-2779-4381-9a92-2ed2a49f8ebc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed4ec5cacb8042149ecd457e00f549e0, user_id=2201178187464803a557d9530be7e0ba, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:12.608232 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a875c6-2779-4381-9a92-2ed2a49f8ebc admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:12.622478 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1096/2176] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:55:12 2022] PUT /identity/v3/projects/ed4ec5cacb8042149ecd457e00f549e0/users/2201178187464803a557d9530be7e0ba/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:55:12.634097 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1133b836-bd54-49cb-bfbc-777f450f4abf None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:12.634736 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1133b836-bd54-49cb-bfbc-777f450f4abf None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:12.635052 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1133b836-bd54-49cb-bfbc-777f450f4abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:12.664958 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-1133b836-bd54-49cb-bfbc-777f450f4abf None None] MFA Rules not processed for user `2201178187464803a557d9530be7e0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:55:12.666201 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-1133b836-bd54-49cb-bfbc-777f450f4abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:12.758824 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1081/2177] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:55:12 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:55:12.794367 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a875c6-2779-4381-9a92-2ed2a49f8ebc admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:12.796198 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-92a875c6-2779-4381-9a92-2ed2a49f8ebc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:12.869498 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a875c6-2779-4381-9a92-2ed2a49f8ebc admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:12.873183 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8833775-48ab-4c62-beec-07f2bfcfcb25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:12.873834 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8833775-48ab-4c62-beec-07f2bfcfcb25 service cinder] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:12.874118 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8833775-48ab-4c62-beec-07f2bfcfcb25 service cinder] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:12.874395 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8833775-48ab-4c62-beec-07f2bfcfcb25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:12.969260 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8833775-48ab-4c62-beec-07f2bfcfcb25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:12.972091 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8833775-48ab-4c62-beec-07f2bfcfcb25 service cinder] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:12.983533 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1097/2178] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:55:12 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:55:14.730943 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:55:15.742188 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:55:16.071835 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:55:17.318814 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:55:17.467090 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:55:18.093189 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:55:18.243865 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:55:18.558146 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:55:19.413963 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:55:19.951420 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1133b836-bd54-49cb-bfbc-777f450f4abf None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:19.968924 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1133b836-bd54-49cb-bfbc-777f450f4abf None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:19.974448 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d33d12-6915-42f6-a394-5cc85e79e1cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:19.975378 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d33d12-6915-42f6-a394-5cc85e79e1cd service cinder] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:19.975824 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d33d12-6915-42f6-a394-5cc85e79e1cd service cinder] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:19.976254 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d33d12-6915-42f6-a394-5cc85e79e1cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:19.990719 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d33d12-6915-42f6-a394-5cc85e79e1cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:19.992496 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d33d12-6915-42f6-a394-5cc85e79e1cd service cinder] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:20.011240 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1082/2179] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:55:19 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:55:24.974581 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8833775-48ab-4c62-beec-07f2bfcfcb25 service cinder] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:24.976850 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-c8833775-48ab-4c62-beec-07f2bfcfcb25 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:25.065299 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8833775-48ab-4c62-beec-07f2bfcfcb25 service cinder] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:25.071177 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf62e6a-59c6-4eb4-ace7-8d5f2ec9d677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:25.072084 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf62e6a-59c6-4eb4-ace7-8d5f2ec9d677 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:25.075639 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf62e6a-59c6-4eb4-ace7-8d5f2ec9d677 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:25.075639 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf62e6a-59c6-4eb4-ace7-8d5f2ec9d677 admin admin] PATH_INFO: `/v3/users/ae84a77e67764396bd2a7564f64df147` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:25.075639 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf62e6a-59c6-4eb4-ace7-8d5f2ec9d677 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae84a77e67764396bd2a7564f64df147)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:25.075975 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf62e6a-59c6-4eb4-ace7-8d5f2ec9d677 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:25.253394 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-cdf62e6a-59c6-4eb4-ace7-8d5f2ec9d677 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae84a77e67764396bd2a7564f64df147'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:25.254840 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:55:25.261520 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-cdf62e6a-59c6-4eb4-ace7-8d5f2ec9d677 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae84a77e67764396bd2a7564f64df147'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:25.262216 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-cdf62e6a-59c6-4eb4-ace7-8d5f2ec9d677 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae84a77e67764396bd2a7564f64df147'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:25.274846 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-cdf62e6a-59c6-4eb4-ace7-8d5f2ec9d677 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae84a77e67764396bd2a7564f64df147'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:25.335143 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-cdf62e6a-59c6-4eb4-ace7-8d5f2ec9d677 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae84a77e67764396bd2a7564f64df147'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:25.341804 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1098/2180] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:55:24 2022] DELETE /identity/v3/users/ae84a77e67764396bd2a7564f64df147 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:55:25.347059 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7648641b-ea46-4a39-a77d-3e908f787dd0 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:25.347554 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7648641b-ea46-4a39-a77d-3e908f787dd0 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:25.347554 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7648641b-ea46-4a39-a77d-3e908f787dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:25.367911 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-7648641b-ea46-4a39-a77d-3e908f787dd0 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:55:25.369165 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-7648641b-ea46-4a39-a77d-3e908f787dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:25.430469 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1083/2181] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:55:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:55:25.441185 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf62e6a-59c6-4eb4-ace7-8d5f2ec9d677 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:25.442817 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-cdf62e6a-59c6-4eb4-ace7-8d5f2ec9d677 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:25.516729 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf62e6a-59c6-4eb4-ace7-8d5f2ec9d677 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:25.520492 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318c113e-ba61-404d-ade8-a2e4b7945ffe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:25.521142 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c113e-ba61-404d-ade8-a2e4b7945ffe service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:25.521425 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c113e-ba61-404d-ade8-a2e4b7945ffe service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:25.521702 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c113e-ba61-404d-ade8-a2e4b7945ffe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:25.530934 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318c113e-ba61-404d-ade8-a2e4b7945ffe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:25.532278 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318c113e-ba61-404d-ade8-a2e4b7945ffe service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:25.543382 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1099/2182] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:55:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:55:26.005436 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7648641b-ea46-4a39-a77d-3e908f787dd0 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:26.007126 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-7648641b-ea46-4a39-a77d-3e908f787dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:26.019320 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7648641b-ea46-4a39-a77d-3e908f787dd0 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:26.024304 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ca8e9-eef5-4f6c-b797-6f3dc15a1ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:26.025230 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ca8e9-eef5-4f6c-b797-6f3dc15a1ba0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:26.025512 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ca8e9-eef5-4f6c-b797-6f3dc15a1ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:26.025787 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ca8e9-eef5-4f6c-b797-6f3dc15a1ba0 admin admin] PATH_INFO: `/v3/projects/ecc2e51ff1354580ae28f9f03f9c916c` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:26.027292 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ca8e9-eef5-4f6c-b797-6f3dc15a1ba0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecc2e51ff1354580ae28f9f03f9c916c)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:26.029219 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ca8e9-eef5-4f6c-b797-6f3dc15a1ba0 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:26.181879 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-997ca8e9-eef5-4f6c-b797-6f3dc15a1ba0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecc2e51ff1354580ae28f9f03f9c916c', 'request_id': 'req-997ca8e9-eef5-4f6c-b797-6f3dc15a1ba0'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:26.183620 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:55:26.188547 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-997ca8e9-eef5-4f6c-b797-6f3dc15a1ba0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecc2e51ff1354580ae28f9f03f9c916c', 'request_id': 'req-997ca8e9-eef5-4f6c-b797-6f3dc15a1ba0'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:26.287110 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-997ca8e9-eef5-4f6c-b797-6f3dc15a1ba0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecc2e51ff1354580ae28f9f03f9c916c', 'request_id': 'req-997ca8e9-eef5-4f6c-b797-6f3dc15a1ba0'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:26.301003 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-997ca8e9-eef5-4f6c-b797-6f3dc15a1ba0 admin admin] The token cache is being invalidate because project ecc2e51ff1354580ae28f9f03f9c916c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:55:26.301312 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-997ca8e9-eef5-4f6c-b797-6f3dc15a1ba0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:26.301982 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-997ca8e9-eef5-4f6c-b797-6f3dc15a1ba0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:26.303799 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1084/2183] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:55:26 2022] DELETE /identity/v3/projects/ecc2e51ff1354580ae28f9f03f9c916c => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:55:35.567658 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=38,keystone:DELETE=3,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:55:36.311998 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:DELETE=3,keystone:SELECT=6,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:55:47.779722 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee9905-e4e2-45ec-a3ee-4ad613cf5623 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:47.779722 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee9905-e4e2-45ec-a3ee-4ad613cf5623 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:47.780359 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee9905-e4e2-45ec-a3ee-4ad613cf5623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:47.806092 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-00ee9905-e4e2-45ec-a3ee-4ad613cf5623 None None] MFA Rules not processed for user `63cc904578554aea84ff1917a5ed075e`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:55:47.807386 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-00ee9905-e4e2-45ec-a3ee-4ad613cf5623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:47.884114 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1100/2184] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jul 21 22:55:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:55:47.939814 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ca8e9-eef5-4f6c-b797-6f3dc15a1ba0 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:47.940402 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-997ca8e9-eef5-4f6c-b797-6f3dc15a1ba0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:48.021833 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ca8e9-eef5-4f6c-b797-6f3dc15a1ba0 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:48.027409 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07875edf-a208-4263-aac3-06f15017ebc5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba7eef4e1b27476ab4016449d9c3a4b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:48.028231 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07875edf-a208-4263-aac3-06f15017ebc5 service swift] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:48.028701 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07875edf-a208-4263-aac3-06f15017ebc5 service swift] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:48.028987 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07875edf-a208-4263-aac3-06f15017ebc5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:48.039496 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07875edf-a208-4263-aac3-06f15017ebc5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:48.041152 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07875edf-a208-4263-aac3-06f15017ebc5 service swift] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:48.050588 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1085/2185] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jul 21 22:55:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 22:55:49.612497 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ee9905-e4e2-45ec-a3ee-4ad613cf5623 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:49.614304 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-00ee9905-e4e2-45ec-a3ee-4ad613cf5623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:49.732988 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ee9905-e4e2-45ec-a3ee-4ad613cf5623 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:49.737873 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419c2c60-3ed3-4521-adf1-0406c9fdef34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:49.739122 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c2c60-3ed3-4521-adf1-0406c9fdef34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:49.739413 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c2c60-3ed3-4521-adf1-0406c9fdef34 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:49.739694 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c2c60-3ed3-4521-adf1-0406c9fdef34 admin admin] PATH_INFO: `/v3/users/b183096b8bdd4f8891612fe72b15c5d1` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:49.741500 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419c2c60-3ed3-4521-adf1-0406c9fdef34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b183096b8bdd4f8891612fe72b15c5d1)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:49.744111 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419c2c60-3ed3-4521-adf1-0406c9fdef34 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:50.015222 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-419c2c60-3ed3-4521-adf1-0406c9fdef34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b183096b8bdd4f8891612fe72b15c5d1'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:50.029559 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-419c2c60-3ed3-4521-adf1-0406c9fdef34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b183096b8bdd4f8891612fe72b15c5d1'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:50.030359 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-419c2c60-3ed3-4521-adf1-0406c9fdef34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b183096b8bdd4f8891612fe72b15c5d1'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:50.054874 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-419c2c60-3ed3-4521-adf1-0406c9fdef34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b183096b8bdd4f8891612fe72b15c5d1'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:50.136029 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-419c2c60-3ed3-4521-adf1-0406c9fdef34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b183096b8bdd4f8891612fe72b15c5d1'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:50.154147 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1101/2186] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:55:49 2022] DELETE /identity/v3/users/b183096b8bdd4f8891612fe72b15c5d1 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:55:50.157716 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b397993-8a38-4635-9547-9340f2984394 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:50.158267 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b397993-8a38-4635-9547-9340f2984394 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:50.158466 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b397993-8a38-4635-9547-9340f2984394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:50.189288 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-5b397993-8a38-4635-9547-9340f2984394 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:55:50.191204 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-5b397993-8a38-4635-9547-9340f2984394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:50.277354 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1086/2187] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:55:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:55:50.289444 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419c2c60-3ed3-4521-adf1-0406c9fdef34 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:50.293052 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-419c2c60-3ed3-4521-adf1-0406c9fdef34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:50.336746 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:55:50.384862 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419c2c60-3ed3-4521-adf1-0406c9fdef34 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:50.387440 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225b85f7-bc7a-4f4e-92cd-5faac393898e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:50.388306 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b85f7-bc7a-4f4e-92cd-5faac393898e service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:50.388693 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b85f7-bc7a-4f4e-92cd-5faac393898e service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:50.389080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b85f7-bc7a-4f4e-92cd-5faac393898e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:50.403404 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225b85f7-bc7a-4f4e-92cd-5faac393898e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:50.405294 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225b85f7-bc7a-4f4e-92cd-5faac393898e service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:50.416735 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1102/2188] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:55:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:55:51.213429 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b397993-8a38-4635-9547-9340f2984394 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:51.216775 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-5b397993-8a38-4635-9547-9340f2984394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:51.240392 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b397993-8a38-4635-9547-9340f2984394 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:51.286298 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d30ea-7ba0-4ba1-ba7c-fa8aa346b839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:51.288483 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d30ea-7ba0-4ba1-ba7c-fa8aa346b839 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:51.288728 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d30ea-7ba0-4ba1-ba7c-fa8aa346b839 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:51.289011 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d30ea-7ba0-4ba1-ba7c-fa8aa346b839 admin admin] PATH_INFO: `/v3/projects/a4d9dacbc48343d38b6a31ff3f2d34a1` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:51.290305 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618d30ea-7ba0-4ba1-ba7c-fa8aa346b839 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4d9dacbc48343d38b6a31ff3f2d34a1)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:51.292266 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618d30ea-7ba0-4ba1-ba7c-fa8aa346b839 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:51.456398 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-618d30ea-7ba0-4ba1-ba7c-fa8aa346b839 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4d9dacbc48343d38b6a31ff3f2d34a1', 'request_id': 'req-618d30ea-7ba0-4ba1-ba7c-fa8aa346b839'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:51.464111 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-618d30ea-7ba0-4ba1-ba7c-fa8aa346b839 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4d9dacbc48343d38b6a31ff3f2d34a1', 'request_id': 'req-618d30ea-7ba0-4ba1-ba7c-fa8aa346b839'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:51.518939 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-618d30ea-7ba0-4ba1-ba7c-fa8aa346b839 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4d9dacbc48343d38b6a31ff3f2d34a1', 'request_id': 'req-618d30ea-7ba0-4ba1-ba7c-fa8aa346b839'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:51.541683 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-618d30ea-7ba0-4ba1-ba7c-fa8aa346b839 admin admin] The token cache is being invalidate because project a4d9dacbc48343d38b6a31ff3f2d34a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:55:51.542164 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-618d30ea-7ba0-4ba1-ba7c-fa8aa346b839 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:51.543309 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-618d30ea-7ba0-4ba1-ba7c-fa8aa346b839 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:51.549550 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1087/2189] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:55:51 2022] DELETE /identity/v3/projects/a4d9dacbc48343d38b6a31ff3f2d34a1 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:55:51.550884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225b85f7-bc7a-4f4e-92cd-5faac393898e service neutron] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:51.562595 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225b85f7-bc7a-4f4e-92cd-5faac393898e service neutron] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:51.641450 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96374302-f791-467d-ada1-f56aa7328122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:51.642525 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96374302-f791-467d-ada1-f56aa7328122 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:51.642906 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96374302-f791-467d-ada1-f56aa7328122 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:51.643322 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96374302-f791-467d-ada1-f56aa7328122 admin admin] PATH_INFO: `/v3/users/2201178187464803a557d9530be7e0ba` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:51.644650 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96374302-f791-467d-ada1-f56aa7328122 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2201178187464803a557d9530be7e0ba)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:51.646843 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96374302-f791-467d-ada1-f56aa7328122 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:51.876014 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-96374302-f791-467d-ada1-f56aa7328122 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2201178187464803a557d9530be7e0ba'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:51.892140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-96374302-f791-467d-ada1-f56aa7328122 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2201178187464803a557d9530be7e0ba'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:51.893136 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-96374302-f791-467d-ada1-f56aa7328122 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2201178187464803a557d9530be7e0ba'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:51.907529 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-96374302-f791-467d-ada1-f56aa7328122 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2201178187464803a557d9530be7e0ba'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:51.975955 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-96374302-f791-467d-ada1-f56aa7328122 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2201178187464803a557d9530be7e0ba'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:51.984419 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1103/2190] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:55:51 2022] DELETE /identity/v3/users/2201178187464803a557d9530be7e0ba => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:55:52.798474 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d30ea-7ba0-4ba1-ba7c-fa8aa346b839 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:52.800455 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-618d30ea-7ba0-4ba1-ba7c-fa8aa346b839 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:52.843799 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d30ea-7ba0-4ba1-ba7c-fa8aa346b839 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:52.867360 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:55:52.931756 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40b9c4b-bf67-42f0-ab2d-fdef908d605e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:52.932931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b9c4b-bf67-42f0-ab2d-fdef908d605e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:52.933434 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b9c4b-bf67-42f0-ab2d-fdef908d605e admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:52.933926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b9c4b-bf67-42f0-ab2d-fdef908d605e admin admin] PATH_INFO: `/v3/projects/ed4ec5cacb8042149ecd457e00f549e0` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:52.935417 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40b9c4b-bf67-42f0-ab2d-fdef908d605e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed4ec5cacb8042149ecd457e00f549e0)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:52.937783 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40b9c4b-bf67-42f0-ab2d-fdef908d605e admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:53.108983 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-c40b9c4b-bf67-42f0-ab2d-fdef908d605e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed4ec5cacb8042149ecd457e00f549e0', 'request_id': 'req-c40b9c4b-bf67-42f0-ab2d-fdef908d605e'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:53.114848 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-c40b9c4b-bf67-42f0-ab2d-fdef908d605e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed4ec5cacb8042149ecd457e00f549e0', 'request_id': 'req-c40b9c4b-bf67-42f0-ab2d-fdef908d605e'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:53.208323 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-c40b9c4b-bf67-42f0-ab2d-fdef908d605e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed4ec5cacb8042149ecd457e00f549e0', 'request_id': 'req-c40b9c4b-bf67-42f0-ab2d-fdef908d605e'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:53.222691 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-c40b9c4b-bf67-42f0-ab2d-fdef908d605e admin admin] The token cache is being invalidate because project ed4ec5cacb8042149ecd457e00f549e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:55:53.223336 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-c40b9c4b-bf67-42f0-ab2d-fdef908d605e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:53.224267 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-c40b9c4b-bf67-42f0-ab2d-fdef908d605e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:55:53.231690 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b578c9-816c-4c50-9346-9ff7e7774c56 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:53.231690 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b578c9-816c-4c50-9346-9ff7e7774c56 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:53.232357 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b578c9-816c-4c50-9346-9ff7e7774c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:53.242204 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1088/2191] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:55:52 2022] DELETE /identity/v3/projects/ed4ec5cacb8042149ecd457e00f549e0 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:55:53.265036 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-84b578c9-816c-4c50-9346-9ff7e7774c56 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:55:53.273030 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-84b578c9-816c-4c50-9346-9ff7e7774c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:53.349253 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:55:53.382573 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1104/2192] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:55:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:55:53.391139 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28cacd5-2152-4b36-bef5-3bdf0e4ff60e None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:53.391553 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28cacd5-2152-4b36-bef5-3bdf0e4ff60e None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:53.391943 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28cacd5-2152-4b36-bef5-3bdf0e4ff60e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:53.441244 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-b28cacd5-2152-4b36-bef5-3bdf0e4ff60e None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:55:53.448841 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-b28cacd5-2152-4b36-bef5-3bdf0e4ff60e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:53.468050 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1089/2193] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:55:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:55:53.472963 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b578c9-816c-4c50-9346-9ff7e7774c56 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:53.488251 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b578c9-816c-4c50-9346-9ff7e7774c56 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:53.500233 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f25e30a-2c67-4915-9470-145369ab17d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:53.500233 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f25e30a-2c67-4915-9470-145369ab17d3 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:53.500570 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f25e30a-2c67-4915-9470-145369ab17d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:53.501003 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f25e30a-2c67-4915-9470-145369ab17d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:53.501917 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f25e30a-2c67-4915-9470-145369ab17d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:55:53.502248 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f25e30a-2c67-4915-9470-145369ab17d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:53.503964 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f25e30a-2c67-4915-9470-145369ab17d3 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:53.522670 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1105/2194] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 22:55:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:55:53.526941 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28cacd5-2152-4b36-bef5-3bdf0e4ff60e None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:53.544201 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28cacd5-2152-4b36-bef5-3bdf0e4ff60e None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:53.549328 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d801752c-1aba-4fcf-98e9-a80258aee8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:53.550107 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d801752c-1aba-4fcf-98e9-a80258aee8e8 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:53.550693 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d801752c-1aba-4fcf-98e9-a80258aee8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:53.551743 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d801752c-1aba-4fcf-98e9-a80258aee8e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:53.552018 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d801752c-1aba-4fcf-98e9-a80258aee8e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:53.554037 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d801752c-1aba-4fcf-98e9-a80258aee8e8 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:53.641721 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1090/2195] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:55:53 2022] POST /identity/v3/projects => generated 375 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:55:53.645912 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f25e30a-2c67-4915-9470-145369ab17d3 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:53.656616 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f25e30a-2c67-4915-9470-145369ab17d3 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:53.735372 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2125c1ae-ffad-424f-875c-1d2f59419730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:53.736117 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125c1ae-ffad-424f-875c-1d2f59419730 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:53.736393 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125c1ae-ffad-424f-875c-1d2f59419730 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:53.736674 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125c1ae-ffad-424f-875c-1d2f59419730 admin admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:53.737370 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2125c1ae-ffad-424f-875c-1d2f59419730 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:53.739373 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2125c1ae-ffad-424f-875c-1d2f59419730 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:53.838481 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1106/2196] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:55:53 2022] POST /identity/v3/users => generated 288 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:55:53.842861 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d801752c-1aba-4fcf-98e9-a80258aee8e8 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:53.854811 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d801752c-1aba-4fcf-98e9-a80258aee8e8 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:53.859977 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07b0bc0-0a41-4167-8e33-75edd66abb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:53.861272 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b0bc0-0a41-4167-8e33-75edd66abb14 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:53.861272 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b0bc0-0a41-4167-8e33-75edd66abb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:53.861781 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b0bc0-0a41-4167-8e33-75edd66abb14 admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:53.862878 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07b0bc0-0a41-4167-8e33-75edd66abb14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:53.864476 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07b0bc0-0a41-4167-8e33-75edd66abb14 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:53.879339 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1091/2197] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:55:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:55:53.881707 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2125c1ae-ffad-424f-875c-1d2f59419730 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:53.909837 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2125c1ae-ffad-424f-875c-1d2f59419730 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:53.919490 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30437e5f-7d3d-4dfb-98c3-e068664ded82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:53.924018 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30437e5f-7d3d-4dfb-98c3-e068664ded82 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:53.924018 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30437e5f-7d3d-4dfb-98c3-e068664ded82 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:53.924018 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30437e5f-7d3d-4dfb-98c3-e068664ded82 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:53.924018 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30437e5f-7d3d-4dfb-98c3-e068664ded82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:53.924018 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30437e5f-7d3d-4dfb-98c3-e068664ded82 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:53.957772 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07b0bc0-0a41-4167-8e33-75edd66abb14 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:53.983859 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1107/2198] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:55:53 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:55:54.006424 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07b0bc0-0a41-4167-8e33-75edd66abb14 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:54.022314 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e14d1aa-4c72-42c4-902f-d06784d2d3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:54.024145 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e14d1aa-4c72-42c4-902f-d06784d2d3ab admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:54.024584 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e14d1aa-4c72-42c4-902f-d06784d2d3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:54.025011 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e14d1aa-4c72-42c4-902f-d06784d2d3ab admin admin] PATH_INFO: `/v3/projects/1fb2ba1b05eb4047a56fb3f1fa4a5569/users/f00d8c41d2ec42ddb2bf720efba040a8/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:54.052161 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e14d1aa-4c72-42c4-902f-d06784d2d3ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fb2ba1b05eb4047a56fb3f1fa4a5569, user_id=f00d8c41d2ec42ddb2bf720efba040a8, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:54.055448 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e14d1aa-4c72-42c4-902f-d06784d2d3ab admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:54.119978 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1092/2199] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:55:53 2022] PUT /identity/v3/projects/1fb2ba1b05eb4047a56fb3f1fa4a5569/users/f00d8c41d2ec42ddb2bf720efba040a8/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:55:54.133699 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad5888-644e-4808-be8c-dcef68363204 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:54.136204 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad5888-644e-4808-be8c-dcef68363204 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:54.136650 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad5888-644e-4808-be8c-dcef68363204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:54.174756 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-08ad5888-644e-4808-be8c-dcef68363204 None None] MFA Rules not processed for user `f00d8c41d2ec42ddb2bf720efba040a8`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:55:54.178864 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-08ad5888-644e-4808-be8c-dcef68363204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:54.307596 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1108/2200] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:55:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:55:54.311811 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e14d1aa-4c72-42c4-902f-d06784d2d3ab admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:54.378846 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:55:54.415057 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e14d1aa-4c72-42c4-902f-d06784d2d3ab admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:54.427583 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4f82b9-5721-4b52-917d-088251b95440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:54.428510 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4f82b9-5721-4b52-917d-088251b95440 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:54.428753 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4f82b9-5721-4b52-917d-088251b95440 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:54.428913 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4f82b9-5721-4b52-917d-088251b95440 admin admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:54.429648 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4f82b9-5721-4b52-917d-088251b95440 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:54.431739 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4f82b9-5721-4b52-917d-088251b95440 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:54.514817 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1093/2201] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:55:54 2022] POST /identity/v3/projects => generated 379 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:55:54.515962 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad5888-644e-4808-be8c-dcef68363204 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:54.527867 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad5888-644e-4808-be8c-dcef68363204 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:54.590892 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:55:54.625428 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613435d1-65fe-4111-b037-212d2e62d251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:54.626138 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613435d1-65fe-4111-b037-212d2e62d251 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:54.626426 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613435d1-65fe-4111-b037-212d2e62d251 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:54.627038 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613435d1-65fe-4111-b037-212d2e62d251 admin admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:54.627932 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613435d1-65fe-4111-b037-212d2e62d251 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:54.629941 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613435d1-65fe-4111-b037-212d2e62d251 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:54.747096 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1109/2202] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:55:54 2022] POST /identity/v3/users => generated 290 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:55:54.751091 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4f82b9-5721-4b52-917d-088251b95440 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:54.764882 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4f82b9-5721-4b52-917d-088251b95440 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:54.770665 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6274ab1-281a-421b-aec9-5b12ca61bf56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:54.771985 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6274ab1-281a-421b-aec9-5b12ca61bf56 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:54.772282 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6274ab1-281a-421b-aec9-5b12ca61bf56 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:54.772554 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6274ab1-281a-421b-aec9-5b12ca61bf56 admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:54.776254 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6274ab1-281a-421b-aec9-5b12ca61bf56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:54.777354 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6274ab1-281a-421b-aec9-5b12ca61bf56 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:54.792238 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1094/2203] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:55:54.796448 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613435d1-65fe-4111-b037-212d2e62d251 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:54.813264 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613435d1-65fe-4111-b037-212d2e62d251 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:54.819606 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023683cd-245c-4cdf-b622-984759e045ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:54.820403 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023683cd-245c-4cdf-b622-984759e045ac admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:54.820792 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023683cd-245c-4cdf-b622-984759e045ac admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:54.821306 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023683cd-245c-4cdf-b622-984759e045ac admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:54.822212 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023683cd-245c-4cdf-b622-984759e045ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:54.823952 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023683cd-245c-4cdf-b622-984759e045ac admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:54.838942 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1110/2204] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:55:54.846699 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6274ab1-281a-421b-aec9-5b12ca61bf56 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:54.859217 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4049c002-9de4-4680-b012-7d8116c7f1c1 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:54.861087 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4049c002-9de4-4680-b012-7d8116c7f1c1 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:54.861087 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4049c002-9de4-4680-b012-7d8116c7f1c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:54.863033 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6274ab1-281a-421b-aec9-5b12ca61bf56 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:54.870521 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bad523-7518-49cf-ba95-b3b0bfe222ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:54.873946 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bad523-7518-49cf-ba95-b3b0bfe222ab admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:54.874563 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bad523-7518-49cf-ba95-b3b0bfe222ab admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:54.875091 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bad523-7518-49cf-ba95-b3b0bfe222ab admin admin] PATH_INFO: `/v3/projects/267d1eb2d3e7404eb3ffc442b0e4a6a3/users/06dc29881802432f87dc5006c6cb0f1a/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:54.894070 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-4049c002-9de4-4680-b012-7d8116c7f1c1 None None] MFA Rules not processed for user `4947466838974a71815561640fa55c14`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:55:54.895344 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-4049c002-9de4-4680-b012-7d8116c7f1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:54.896248 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bad523-7518-49cf-ba95-b3b0bfe222ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=267d1eb2d3e7404eb3ffc442b0e4a6a3, user_id=06dc29881802432f87dc5006c6cb0f1a, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:54.899765 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bad523-7518-49cf-ba95-b3b0bfe222ab admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:54.983145 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1095/2205] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:55:54 2022] PUT /identity/v3/projects/267d1eb2d3e7404eb3ffc442b0e4a6a3/users/06dc29881802432f87dc5006c6cb0f1a/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:55:54.986883 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30799de9-d8a0-4fbd-bd34-5b09b6a6fd62 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:54.986883 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30799de9-d8a0-4fbd-bd34-5b09b6a6fd62 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:54.986883 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30799de9-d8a0-4fbd-bd34-5b09b6a6fd62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:55.018665 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1111/2206] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:55:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:55:55.025169 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-30799de9-d8a0-4fbd-bd34-5b09b6a6fd62 None None] MFA Rules not processed for user `06dc29881802432f87dc5006c6cb0f1a`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:55:55.026456 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-30799de9-d8a0-4fbd-bd34-5b09b6a6fd62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:55.111588 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1096/2207] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:55:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:55:55.116307 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4049c002-9de4-4680-b012-7d8116c7f1c1 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:55.137070 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4049c002-9de4-4680-b012-7d8116c7f1c1 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:55.218415 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dc1ba5-0214-4d85-84b5-a2a172c03551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:55.219285 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc1ba5-0214-4d85-84b5-a2a172c03551 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:55.219673 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc1ba5-0214-4d85-84b5-a2a172c03551 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:55.219990 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc1ba5-0214-4d85-84b5-a2a172c03551 admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:55.220823 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dc1ba5-0214-4d85-84b5-a2a172c03551 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:55.222845 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dc1ba5-0214-4d85-84b5-a2a172c03551 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:55.318859 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1112/2208] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:55:55 2022] POST /identity/v3/projects => generated 377 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:55:55.319533 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30799de9-d8a0-4fbd-bd34-5b09b6a6fd62 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:55.335921 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30799de9-d8a0-4fbd-bd34-5b09b6a6fd62 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:55.405505 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3a1292-58be-40a2-b0ca-29933d763fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:55.406191 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3a1292-58be-40a2-b0ca-29933d763fff admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:55.406483 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3a1292-58be-40a2-b0ca-29933d763fff admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:55.406762 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3a1292-58be-40a2-b0ca-29933d763fff admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:55.407594 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3a1292-58be-40a2-b0ca-29933d763fff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:55.409576 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3a1292-58be-40a2-b0ca-29933d763fff admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:55.517774 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1097/2209] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:55:55 2022] POST /identity/v3/users => generated 295 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:55:55.519328 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dc1ba5-0214-4d85-84b5-a2a172c03551 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:55.532533 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dc1ba5-0214-4d85-84b5-a2a172c03551 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:55.538201 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c064120-fcda-45f9-8726-f725d0d93860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:55.538875 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c064120-fcda-45f9-8726-f725d0d93860 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:55.539162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c064120-fcda-45f9-8726-f725d0d93860 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:55.539473 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c064120-fcda-45f9-8726-f725d0d93860 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:55.540215 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c064120-fcda-45f9-8726-f725d0d93860 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:55.541822 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c064120-fcda-45f9-8726-f725d0d93860 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:55.557267 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1113/2210] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:55:55.558464 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3a1292-58be-40a2-b0ca-29933d763fff admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:55.569251 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3a1292-58be-40a2-b0ca-29933d763fff admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:55.576719 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f40a2bb-9f81-46ec-9db6-c0a559044cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:55.577701 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40a2bb-9f81-46ec-9db6-c0a559044cda admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:55.578007 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40a2bb-9f81-46ec-9db6-c0a559044cda admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:55.578287 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40a2bb-9f81-46ec-9db6-c0a559044cda admin admin] PATH_INFO: `/v3/projects/8bd5d8d347ca409e813e7afd4301ed58/users/3a23de8cf7ef4c8f8f563f2c3d679995/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:55.583329 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:55:55.596926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f40a2bb-9f81-46ec-9db6-c0a559044cda admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bd5d8d347ca409e813e7afd4301ed58, user_id=3a23de8cf7ef4c8f8f563f2c3d679995, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:55.600290 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f40a2bb-9f81-46ec-9db6-c0a559044cda admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:55.674907 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1098/2211] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:55:55 2022] PUT /identity/v3/projects/8bd5d8d347ca409e813e7afd4301ed58/users/3a23de8cf7ef4c8f8f563f2c3d679995/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:55:55.679110 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c064120-fcda-45f9-8726-f725d0d93860 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:55.690400 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c064120-fcda-45f9-8726-f725d0d93860 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:55.746341 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:55:55.766338 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13450d9a-162c-4cdd-908a-3ead8881c5b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:55.766338 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13450d9a-162c-4cdd-908a-3ead8881c5b5 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:55.766649 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13450d9a-162c-4cdd-908a-3ead8881c5b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:55.766965 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13450d9a-162c-4cdd-908a-3ead8881c5b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:55.767776 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13450d9a-162c-4cdd-908a-3ead8881c5b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:55.769422 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13450d9a-162c-4cdd-908a-3ead8881c5b5 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:55.781075 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1114/2212] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:55:55.787460 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f40a2bb-9f81-46ec-9db6-c0a559044cda admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:55.809599 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f40a2bb-9f81-46ec-9db6-c0a559044cda admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:55.817190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b9bc8b-822d-4337-a2f1-19f0aef65b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:55.819148 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b9bc8b-822d-4337-a2f1-19f0aef65b56 admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:55.819445 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b9bc8b-822d-4337-a2f1-19f0aef65b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:55.819757 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b9bc8b-822d-4337-a2f1-19f0aef65b56 admin admin] PATH_INFO: `/v3/projects/8bd5d8d347ca409e813e7afd4301ed58/users/3a23de8cf7ef4c8f8f563f2c3d679995/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:55.821884 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b9bc8b-822d-4337-a2f1-19f0aef65b56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bd5d8d347ca409e813e7afd4301ed58, user_id=3a23de8cf7ef4c8f8f563f2c3d679995, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:55.824742 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b9bc8b-822d-4337-a2f1-19f0aef65b56 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:55.839004 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1099/2213] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:55:55 2022] PUT /identity/v3/projects/8bd5d8d347ca409e813e7afd4301ed58/users/3a23de8cf7ef4c8f8f563f2c3d679995/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:55:55.854165 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3127d280-8a22-4a06-bc0e-6c7143b82272 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:55.854165 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3127d280-8a22-4a06-bc0e-6c7143b82272 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:55.854165 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3127d280-8a22-4a06-bc0e-6c7143b82272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:55.886245 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-3127d280-8a22-4a06-bc0e-6c7143b82272 None None] MFA Rules not processed for user `3a23de8cf7ef4c8f8f563f2c3d679995`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:55:55.887921 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-3127d280-8a22-4a06-bc0e-6c7143b82272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:55.973123 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1115/2214] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:55:55 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:55:56.070415 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b9bc8b-822d-4337-a2f1-19f0aef65b56 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:56.072269 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-14b9bc8b-822d-4337-a2f1-19f0aef65b56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:56.149611 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b9bc8b-822d-4337-a2f1-19f0aef65b56 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:56.159739 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a64d8a-a2eb-408d-8ef5-af5da8d12fd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:56.160492 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a64d8a-a2eb-408d-8ef5-af5da8d12fd1 service cinder] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:56.160850 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a64d8a-a2eb-408d-8ef5-af5da8d12fd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:56.161250 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a64d8a-a2eb-408d-8ef5-af5da8d12fd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:56.179026 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a64d8a-a2eb-408d-8ef5-af5da8d12fd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:56.179682 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a64d8a-a2eb-408d-8ef5-af5da8d12fd1 service cinder] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:56.190464 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1100/2215] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:55:56 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:55:56.367558 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3127d280-8a22-4a06-bc0e-6c7143b82272 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:55:56.369037 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-3127d280-8a22-4a06-bc0e-6c7143b82272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:55:56.384666 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3127d280-8a22-4a06-bc0e-6c7143b82272 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:55:56.385030 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951917cc-74d6-47e6-8da3-4bfd8ed6de8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:55:56.385695 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951917cc-74d6-47e6-8da3-4bfd8ed6de8b service cinder] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:55:56.385980 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951917cc-74d6-47e6-8da3-4bfd8ed6de8b service cinder] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:55:56.386260 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951917cc-74d6-47e6-8da3-4bfd8ed6de8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:55:56.446878 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951917cc-74d6-47e6-8da3-4bfd8ed6de8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:55:56.448860 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951917cc-74d6-47e6-8da3-4bfd8ed6de8b service cinder] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:55:56.462762 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1116/2216] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:55:56 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:56:01.252298 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a64d8a-a2eb-408d-8ef5-af5da8d12fd1 service cinder] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:56:01.320915 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a64d8a-a2eb-408d-8ef5-af5da8d12fd1 service cinder] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:56:01.325613 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5225ca08-c3ca-4f2d-9367-26d63fa8688f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:56:01.326512 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225ca08-c3ca-4f2d-9367-26d63fa8688f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:01.326816 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225ca08-c3ca-4f2d-9367-26d63fa8688f admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:01.327096 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225ca08-c3ca-4f2d-9367-26d63fa8688f admin admin] PATH_INFO: `/v3/users/f00d8c41d2ec42ddb2bf720efba040a8` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:01.328351 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5225ca08-c3ca-4f2d-9367-26d63fa8688f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f00d8c41d2ec42ddb2bf720efba040a8)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:56:01.332589 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5225ca08-c3ca-4f2d-9367-26d63fa8688f admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:56:01.540810 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-5225ca08-c3ca-4f2d-9367-26d63fa8688f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f00d8c41d2ec42ddb2bf720efba040a8'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:56:01.549084 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-5225ca08-c3ca-4f2d-9367-26d63fa8688f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f00d8c41d2ec42ddb2bf720efba040a8'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:56:01.549898 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-5225ca08-c3ca-4f2d-9367-26d63fa8688f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f00d8c41d2ec42ddb2bf720efba040a8'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:56:01.566211 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-5225ca08-c3ca-4f2d-9367-26d63fa8688f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f00d8c41d2ec42ddb2bf720efba040a8'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:56:01.567124 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-5225ca08-c3ca-4f2d-9367-26d63fa8688f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f00d8c41d2ec42ddb2bf720efba040a8'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:56:01.570930 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:56:01.640535 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1101/2217] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:56:01 2022] DELETE /identity/v3/users/f00d8c41d2ec42ddb2bf720efba040a8 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:56:01.645099 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f78bdf-3329-41fd-abed-33e684e0824f None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:01.645634 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f78bdf-3329-41fd-abed-33e684e0824f None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:01.645817 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f78bdf-3329-41fd-abed-33e684e0824f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:01.665959 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-19f78bdf-3329-41fd-abed-33e684e0824f None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:56:01.667236 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-19f78bdf-3329-41fd-abed-33e684e0824f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:56:01.725019 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:56:01.733920 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1117/2218] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:56:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:56:01.744412 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5225ca08-c3ca-4f2d-9367-26d63fa8688f admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:56:01.745933 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-5225ca08-c3ca-4f2d-9367-26d63fa8688f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:56:01.804673 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5225ca08-c3ca-4f2d-9367-26d63fa8688f admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:56:01.808486 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9ba1eb-72fe-46c9-95ed-f246e3eed29a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:56:01.813301 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9ba1eb-72fe-46c9-95ed-f246e3eed29a service neutron] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:01.813301 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9ba1eb-72fe-46c9-95ed-f246e3eed29a service neutron] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:01.813301 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9ba1eb-72fe-46c9-95ed-f246e3eed29a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:01.820783 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9ba1eb-72fe-46c9-95ed-f246e3eed29a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:56:01.822222 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9ba1eb-72fe-46c9-95ed-f246e3eed29a service neutron] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:56:01.832109 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1102/2219] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:56:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:56:02.718479 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f78bdf-3329-41fd-abed-33e684e0824f None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:56:02.721796 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-19f78bdf-3329-41fd-abed-33e684e0824f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:56:02.735699 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f78bdf-3329-41fd-abed-33e684e0824f None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:56:02.740867 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee619e-ac35-42c7-8397-83073c7570a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:56:02.741348 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee619e-ac35-42c7-8397-83073c7570a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:02.741677 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee619e-ac35-42c7-8397-83073c7570a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:02.741942 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee619e-ac35-42c7-8397-83073c7570a4 admin admin] PATH_INFO: `/v3/projects/1fb2ba1b05eb4047a56fb3f1fa4a5569` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:02.743185 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ee619e-ac35-42c7-8397-83073c7570a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fb2ba1b05eb4047a56fb3f1fa4a5569)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:56:02.745114 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ee619e-ac35-42c7-8397-83073c7570a4 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:56:02.873618 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-44ee619e-ac35-42c7-8397-83073c7570a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fb2ba1b05eb4047a56fb3f1fa4a5569', 'request_id': 'req-44ee619e-ac35-42c7-8397-83073c7570a4'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:56:02.883622 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-44ee619e-ac35-42c7-8397-83073c7570a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fb2ba1b05eb4047a56fb3f1fa4a5569', 'request_id': 'req-44ee619e-ac35-42c7-8397-83073c7570a4'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:56:02.959923 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-44ee619e-ac35-42c7-8397-83073c7570a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fb2ba1b05eb4047a56fb3f1fa4a5569', 'request_id': 'req-44ee619e-ac35-42c7-8397-83073c7570a4'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:56:02.973114 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-44ee619e-ac35-42c7-8397-83073c7570a4 admin admin] The token cache is being invalidate because project 1fb2ba1b05eb4047a56fb3f1fa4a5569 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:56:02.973402 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-44ee619e-ac35-42c7-8397-83073c7570a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:56:02.974264 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-44ee619e-ac35-42c7-8397-83073c7570a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:56:02.975995 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1118/2220] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:56:02 2022] DELETE /identity/v3/projects/1fb2ba1b05eb4047a56fb3f1fa4a5569 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:56:02.980121 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9ba1eb-72fe-46c9-95ed-f246e3eed29a service neutron] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:56:02.991094 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9ba1eb-72fe-46c9-95ed-f246e3eed29a service neutron] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:56:03.053742 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1194f31a-98f9-40f5-9a2c-fbd59b8a624a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:56:03.054596 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194f31a-98f9-40f5-9a2c-fbd59b8a624a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:03.054905 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194f31a-98f9-40f5-9a2c-fbd59b8a624a admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:03.055179 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194f31a-98f9-40f5-9a2c-fbd59b8a624a admin admin] PATH_INFO: `/v3/users/06dc29881802432f87dc5006c6cb0f1a` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:03.056362 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1194f31a-98f9-40f5-9a2c-fbd59b8a624a admin admin] RBAC: Authorizing `identity:delete_user(user_id=06dc29881802432f87dc5006c6cb0f1a)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:56:03.058354 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1194f31a-98f9-40f5-9a2c-fbd59b8a624a admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:56:03.281824 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-1194f31a-98f9-40f5-9a2c-fbd59b8a624a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06dc29881802432f87dc5006c6cb0f1a'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:56:03.289731 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-1194f31a-98f9-40f5-9a2c-fbd59b8a624a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06dc29881802432f87dc5006c6cb0f1a'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:56:03.290640 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-1194f31a-98f9-40f5-9a2c-fbd59b8a624a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06dc29881802432f87dc5006c6cb0f1a'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:56:03.310883 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-1194f31a-98f9-40f5-9a2c-fbd59b8a624a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06dc29881802432f87dc5006c6cb0f1a'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:56:03.312691 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-1194f31a-98f9-40f5-9a2c-fbd59b8a624a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06dc29881802432f87dc5006c6cb0f1a'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:56:03.379902 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1103/2221] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:56:02 2022] DELETE /identity/v3/users/06dc29881802432f87dc5006c6cb0f1a => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:56:03.860897 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee619e-ac35-42c7-8397-83073c7570a4 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:56:03.862405 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-44ee619e-ac35-42c7-8397-83073c7570a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:56:03.875621 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee619e-ac35-42c7-8397-83073c7570a4 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:56:03.932010 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3afa53-cb46-4c61-a14b-1e5d763f0694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:56:03.932880 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3afa53-cb46-4c61-a14b-1e5d763f0694 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:03.933167 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3afa53-cb46-4c61-a14b-1e5d763f0694 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:03.933439 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3afa53-cb46-4c61-a14b-1e5d763f0694 admin admin] PATH_INFO: `/v3/projects/267d1eb2d3e7404eb3ffc442b0e4a6a3` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:03.934631 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3afa53-cb46-4c61-a14b-1e5d763f0694 admin admin] RBAC: Authorizing `identity:delete_project(project_id=267d1eb2d3e7404eb3ffc442b0e4a6a3)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:56:03.936635 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3afa53-cb46-4c61-a14b-1e5d763f0694 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:56:04.055533 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-dd3afa53-cb46-4c61-a14b-1e5d763f0694 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '267d1eb2d3e7404eb3ffc442b0e4a6a3', 'request_id': 'req-dd3afa53-cb46-4c61-a14b-1e5d763f0694'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:56:04.061573 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-dd3afa53-cb46-4c61-a14b-1e5d763f0694 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '267d1eb2d3e7404eb3ffc442b0e4a6a3', 'request_id': 'req-dd3afa53-cb46-4c61-a14b-1e5d763f0694'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:56:04.136171 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-dd3afa53-cb46-4c61-a14b-1e5d763f0694 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '267d1eb2d3e7404eb3ffc442b0e4a6a3', 'request_id': 'req-dd3afa53-cb46-4c61-a14b-1e5d763f0694'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:56:04.149302 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-dd3afa53-cb46-4c61-a14b-1e5d763f0694 admin admin] The token cache is being invalidate because project 267d1eb2d3e7404eb3ffc442b0e4a6a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:56:04.149302 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-dd3afa53-cb46-4c61-a14b-1e5d763f0694 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:56:04.149940 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-dd3afa53-cb46-4c61-a14b-1e5d763f0694 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:56:04.151531 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1119/2222] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:56:03 2022] DELETE /identity/v3/projects/267d1eb2d3e7404eb3ffc442b0e4a6a3 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:56:04.155502 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1194f31a-98f9-40f5-9a2c-fbd59b8a624a admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:56:04.156945 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-1194f31a-98f9-40f5-9a2c-fbd59b8a624a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:56:04.170607 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1194f31a-98f9-40f5-9a2c-fbd59b8a624a admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:56:04.184295 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:56:04.232309 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c8fd4-9d54-4d0e-90d2-078aaf20d698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:56:04.233195 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c8fd4-9d54-4d0e-90d2-078aaf20d698 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:04.233485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c8fd4-9d54-4d0e-90d2-078aaf20d698 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:04.233767 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c8fd4-9d54-4d0e-90d2-078aaf20d698 admin admin] PATH_INFO: `/v3/users/3a23de8cf7ef4c8f8f563f2c3d679995` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:04.235106 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902c8fd4-9d54-4d0e-90d2-078aaf20d698 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a23de8cf7ef4c8f8f563f2c3d679995)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:56:04.237033 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902c8fd4-9d54-4d0e-90d2-078aaf20d698 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:56:04.442602 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-902c8fd4-9d54-4d0e-90d2-078aaf20d698 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a23de8cf7ef4c8f8f563f2c3d679995'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:56:04.450429 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-902c8fd4-9d54-4d0e-90d2-078aaf20d698 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a23de8cf7ef4c8f8f563f2c3d679995'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:56:04.451155 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-902c8fd4-9d54-4d0e-90d2-078aaf20d698 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a23de8cf7ef4c8f8f563f2c3d679995'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:56:04.464999 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-902c8fd4-9d54-4d0e-90d2-078aaf20d698 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a23de8cf7ef4c8f8f563f2c3d679995'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:56:04.465702 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-902c8fd4-9d54-4d0e-90d2-078aaf20d698 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a23de8cf7ef4c8f8f563f2c3d679995'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:56:04.534538 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1104/2223] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:56:04 2022] DELETE /identity/v3/users/3a23de8cf7ef4c8f8f563f2c3d679995 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:56:05.061513 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3afa53-cb46-4c61-a14b-1e5d763f0694 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:56:05.063047 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-dd3afa53-cb46-4c61-a14b-1e5d763f0694 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:56:05.078025 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3afa53-cb46-4c61-a14b-1e5d763f0694 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:56:05.088353 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:56:05.137070 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0d01fc-5342-41ba-8ddd-8e7041bd4b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:56:05.137960 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d01fc-5342-41ba-8ddd-8e7041bd4b3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:05.138285 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d01fc-5342-41ba-8ddd-8e7041bd4b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:05.138558 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d01fc-5342-41ba-8ddd-8e7041bd4b3b admin admin] PATH_INFO: `/v3/projects/8bd5d8d347ca409e813e7afd4301ed58` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:05.139811 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0d01fc-5342-41ba-8ddd-8e7041bd4b3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bd5d8d347ca409e813e7afd4301ed58)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:56:05.141758 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0d01fc-5342-41ba-8ddd-8e7041bd4b3b admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:56:05.234806 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-af0d01fc-5342-41ba-8ddd-8e7041bd4b3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bd5d8d347ca409e813e7afd4301ed58', 'request_id': 'req-af0d01fc-5342-41ba-8ddd-8e7041bd4b3b'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:56:05.240405 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-af0d01fc-5342-41ba-8ddd-8e7041bd4b3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bd5d8d347ca409e813e7afd4301ed58', 'request_id': 'req-af0d01fc-5342-41ba-8ddd-8e7041bd4b3b'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:56:05.302135 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-af0d01fc-5342-41ba-8ddd-8e7041bd4b3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bd5d8d347ca409e813e7afd4301ed58', 'request_id': 'req-af0d01fc-5342-41ba-8ddd-8e7041bd4b3b'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:56:05.317273 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-af0d01fc-5342-41ba-8ddd-8e7041bd4b3b admin admin] The token cache is being invalidate because project 8bd5d8d347ca409e813e7afd4301ed58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:56:05.317591 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-af0d01fc-5342-41ba-8ddd-8e7041bd4b3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:56:05.318253 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-af0d01fc-5342-41ba-8ddd-8e7041bd4b3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:56:05.319917 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1120/2224] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:56:05 2022] DELETE /identity/v3/projects/8bd5d8d347ca409e813e7afd4301ed58 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:56:05.325813 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee835b4-4ac5-4539-9050-b00386812035 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:05.326251 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee835b4-4ac5-4539-9050-b00386812035 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:05.326379 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee835b4-4ac5-4539-9050-b00386812035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:05.346334 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-fee835b4-4ac5-4539-9050-b00386812035 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:56:05.347574 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-fee835b4-4ac5-4539-9050-b00386812035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:56:05.411437 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1105/2225] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:56:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:56:05.420522 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b10bd-9fc4-4a4a-ae0c-2248a8727934 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:05.420959 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b10bd-9fc4-4a4a-ae0c-2248a8727934 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:05.421247 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b10bd-9fc4-4a4a-ae0c-2248a8727934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:05.440849 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-865b10bd-9fc4-4a4a-ae0c-2248a8727934 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:56:05.442142 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-865b10bd-9fc4-4a4a-ae0c-2248a8727934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:56:05.450485 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1121/2226] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:56:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:56:05.454863 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee835b4-4ac5-4539-9050-b00386812035 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:56:05.466100 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee835b4-4ac5-4539-9050-b00386812035 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:56:05.471106 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5258d1c3-b4d2-4803-bd0e-e067ee9661b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:56:05.471773 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5258d1c3-b4d2-4803-bd0e-e067ee9661b4 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:05.472142 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5258d1c3-b4d2-4803-bd0e-e067ee9661b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:05.472465 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5258d1c3-b4d2-4803-bd0e-e067ee9661b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:05.473170 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5258d1c3-b4d2-4803-bd0e-e067ee9661b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:56:05.473478 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5258d1c3-b4d2-4803-bd0e-e067ee9661b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:56:05.481143 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5258d1c3-b4d2-4803-bd0e-e067ee9661b4 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:56:05.494344 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1106/2227] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 22:56:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:56:05.498255 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865b10bd-9fc4-4a4a-ae0c-2248a8727934 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:56:05.507948 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865b10bd-9fc4-4a4a-ae0c-2248a8727934 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:56:05.514232 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18f7592-f5e2-4185-a837-63a5d86b463b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:56:05.514232 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18f7592-f5e2-4185-a837-63a5d86b463b admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:05.514232 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18f7592-f5e2-4185-a837-63a5d86b463b admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:05.514554 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18f7592-f5e2-4185-a837-63a5d86b463b admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:05.515161 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18f7592-f5e2-4185-a837-63a5d86b463b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:56:05.517066 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18f7592-f5e2-4185-a837-63a5d86b463b admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:56:05.615847 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1122/2228] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:56:05 2022] POST /identity/v3/projects => generated 371 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:56:05.617951 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5258d1c3-b4d2-4803-bd0e-e067ee9661b4 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:56:05.628014 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5258d1c3-b4d2-4803-bd0e-e067ee9661b4 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:56:05.647628 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:56:05.686046 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0521a6-786b-4051-8158-e31243ed864f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:56:05.686721 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0521a6-786b-4051-8158-e31243ed864f admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:05.687031 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0521a6-786b-4051-8158-e31243ed864f admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:05.687307 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0521a6-786b-4051-8158-e31243ed864f admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:05.688001 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0521a6-786b-4051-8158-e31243ed864f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:56:05.689891 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0521a6-786b-4051-8158-e31243ed864f admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:56:05.757981 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1107/2229] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:56:05 2022] POST /identity/v3/users => generated 286 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:56:05.762207 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18f7592-f5e2-4185-a837-63a5d86b463b admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:56:05.773682 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18f7592-f5e2-4185-a837-63a5d86b463b admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:56:05.778728 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aed831-04ae-475c-b22d-07dc28b1507c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:56:05.779464 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aed831-04ae-475c-b22d-07dc28b1507c admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:05.779746 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aed831-04ae-475c-b22d-07dc28b1507c admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:05.780025 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aed831-04ae-475c-b22d-07dc28b1507c admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:05.780805 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aed831-04ae-475c-b22d-07dc28b1507c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:56:05.782423 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aed831-04ae-475c-b22d-07dc28b1507c admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:56:05.793504 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1123/2230] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:56:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:56:05.797842 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0521a6-786b-4051-8158-e31243ed864f admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:56:05.807986 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0521a6-786b-4051-8158-e31243ed864f admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:56:05.813074 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d32475e-7d6f-4adf-9d42-9a82de426d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:56:05.813820 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d32475e-7d6f-4adf-9d42-9a82de426d5f admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:05.814105 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d32475e-7d6f-4adf-9d42-9a82de426d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:05.814394 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d32475e-7d6f-4adf-9d42-9a82de426d5f admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:05.815382 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d32475e-7d6f-4adf-9d42-9a82de426d5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:56:05.817395 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d32475e-7d6f-4adf-9d42-9a82de426d5f admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:56:05.829035 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1108/2231] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:56:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:56:05.833083 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aed831-04ae-475c-b22d-07dc28b1507c admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:56:05.844134 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aed831-04ae-475c-b22d-07dc28b1507c admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:56:05.849346 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46b6cc4-bcfc-4833-a66e-3bae2e7afcf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:56:05.850256 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b6cc4-bcfc-4833-a66e-3bae2e7afcf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:05.850542 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b6cc4-bcfc-4833-a66e-3bae2e7afcf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:05.850856 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b6cc4-bcfc-4833-a66e-3bae2e7afcf1 admin admin] PATH_INFO: `/v3/projects/7526d4f831844eae8549777bfb42639f/users/2f63fcfac3744a42b24898eb3a886d2c/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:05.867126 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46b6cc4-bcfc-4833-a66e-3bae2e7afcf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7526d4f831844eae8549777bfb42639f, user_id=2f63fcfac3744a42b24898eb3a886d2c, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:56:05.869786 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46b6cc4-bcfc-4833-a66e-3bae2e7afcf1 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:56:05.959534 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1124/2232] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:56:05 2022] PUT /identity/v3/projects/7526d4f831844eae8549777bfb42639f/users/2f63fcfac3744a42b24898eb3a886d2c/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:56:05.972582 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12907c6-9ccb-4849-90fe-a91e51a65ed4 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:05.972869 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12907c6-9ccb-4849-90fe-a91e51a65ed4 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:05.973154 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12907c6-9ccb-4849-90fe-a91e51a65ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:06.000359 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-f12907c6-9ccb-4849-90fe-a91e51a65ed4 None None] MFA Rules not processed for user `2f63fcfac3744a42b24898eb3a886d2c`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:56:06.001686 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-f12907c6-9ccb-4849-90fe-a91e51a65ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:56:06.068127 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1109/2233] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:56:05 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:56:06.079893 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46b6cc4-bcfc-4833-a66e-3bae2e7afcf1 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:56:06.081723 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-a46b6cc4-bcfc-4833-a66e-3bae2e7afcf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:56:06.124623 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:56:06.139257 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46b6cc4-bcfc-4833-a66e-3bae2e7afcf1 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:56:06.142905 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35067ff-1247-4b5a-a9b0-b96bde23abfc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:56:06.143589 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35067ff-1247-4b5a-a9b0-b96bde23abfc service cinder] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:06.143875 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35067ff-1247-4b5a-a9b0-b96bde23abfc service cinder] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:06.144156 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35067ff-1247-4b5a-a9b0-b96bde23abfc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:06.153943 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35067ff-1247-4b5a-a9b0-b96bde23abfc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:56:06.155443 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35067ff-1247-4b5a-a9b0-b96bde23abfc service cinder] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:56:06.164455 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1125/2234] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:56:06 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:56:06.188946 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12907c6-9ccb-4849-90fe-a91e51a65ed4 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:56:06.210586 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35067ff-1247-4b5a-a9b0-b96bde23abfc service cinder] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:56:06.212041 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-d35067ff-1247-4b5a-a9b0-b96bde23abfc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:56:06.266576 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12907c6-9ccb-4849-90fe-a91e51a65ed4 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:56:06.272405 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19be31b-98bc-430b-ae6f-475def8c552a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:56:06.273458 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19be31b-98bc-430b-ae6f-475def8c552a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:06.273745 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19be31b-98bc-430b-ae6f-475def8c552a admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:06.274031 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19be31b-98bc-430b-ae6f-475def8c552a admin admin] PATH_INFO: `/v3/users/2f63fcfac3744a42b24898eb3a886d2c` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:06.275282 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19be31b-98bc-430b-ae6f-475def8c552a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f63fcfac3744a42b24898eb3a886d2c)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:56:06.277225 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19be31b-98bc-430b-ae6f-475def8c552a admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:56:06.290636 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35067ff-1247-4b5a-a9b0-b96bde23abfc service cinder] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:56:06.294340 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4afa93f-037c-496c-8224-395c3608ff21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:56:06.295013 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4afa93f-037c-496c-8224-395c3608ff21 service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:06.295294 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4afa93f-037c-496c-8224-395c3608ff21 service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:06.295588 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4afa93f-037c-496c-8224-395c3608ff21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:06.297400 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-a4afa93f-037c-496c-8224-395c3608ff21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:56:06.364185 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4afa93f-037c-496c-8224-395c3608ff21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:56:06.365670 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4afa93f-037c-496c-8224-395c3608ff21 service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:56:06.375816 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1126/2235] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:56:06 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:56:06.427258 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:56:06.439920 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-a19be31b-98bc-430b-ae6f-475def8c552a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f63fcfac3744a42b24898eb3a886d2c'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:56:06.449021 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-a19be31b-98bc-430b-ae6f-475def8c552a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f63fcfac3744a42b24898eb3a886d2c'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:56:06.449702 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-a19be31b-98bc-430b-ae6f-475def8c552a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f63fcfac3744a42b24898eb3a886d2c'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:56:06.461678 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-a19be31b-98bc-430b-ae6f-475def8c552a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f63fcfac3744a42b24898eb3a886d2c'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:56:06.462326 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-a19be31b-98bc-430b-ae6f-475def8c552a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f63fcfac3744a42b24898eb3a886d2c'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:56:06.540281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1110/2236] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:56:06 2022] DELETE /identity/v3/users/2f63fcfac3744a42b24898eb3a886d2c => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:56:06.545647 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a073fb7-b782-4b1e-ab12-515659f27cfc None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:06.545944 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a073fb7-b782-4b1e-ab12-515659f27cfc None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:06.546274 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a073fb7-b782-4b1e-ab12-515659f27cfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:06.565338 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-5a073fb7-b782-4b1e-ab12-515659f27cfc None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:56:06.566562 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-5a073fb7-b782-4b1e-ab12-515659f27cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:56:06.628266 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1127/2237] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:56:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:56:06.640123 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19be31b-98bc-430b-ae6f-475def8c552a admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:56:06.641745 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-a19be31b-98bc-430b-ae6f-475def8c552a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:56:06.705036 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19be31b-98bc-430b-ae6f-475def8c552a admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:56:06.708714 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2adbe2f-0a9e-4fe9-affd-320c159ca0a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:56:06.709495 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2adbe2f-0a9e-4fe9-affd-320c159ca0a7 service neutron] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:06.709826 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2adbe2f-0a9e-4fe9-affd-320c159ca0a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:06.710107 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2adbe2f-0a9e-4fe9-affd-320c159ca0a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:06.719914 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2adbe2f-0a9e-4fe9-affd-320c159ca0a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:56:06.721367 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2adbe2f-0a9e-4fe9-affd-320c159ca0a7 service neutron] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:56:06.732270 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1111/2238] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:56:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:56:07.601573 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a073fb7-b782-4b1e-ab12-515659f27cfc None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:56:07.603139 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-5a073fb7-b782-4b1e-ab12-515659f27cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:56:07.618636 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a073fb7-b782-4b1e-ab12-515659f27cfc None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:56:07.631943 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede8b5ed-98ae-4b81-834f-62b908a6e51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:56:07.631943 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede8b5ed-98ae-4b81-834f-62b908a6e51e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:07.631943 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede8b5ed-98ae-4b81-834f-62b908a6e51e admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:07.631943 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede8b5ed-98ae-4b81-834f-62b908a6e51e admin admin] PATH_INFO: `/v3/projects/7526d4f831844eae8549777bfb42639f` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:07.635222 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede8b5ed-98ae-4b81-834f-62b908a6e51e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7526d4f831844eae8549777bfb42639f)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:56:07.637526 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede8b5ed-98ae-4b81-834f-62b908a6e51e admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:56:07.746179 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:56:07.763191 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-ede8b5ed-98ae-4b81-834f-62b908a6e51e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7526d4f831844eae8549777bfb42639f', 'request_id': 'req-ede8b5ed-98ae-4b81-834f-62b908a6e51e'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:56:07.768641 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-ede8b5ed-98ae-4b81-834f-62b908a6e51e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7526d4f831844eae8549777bfb42639f', 'request_id': 'req-ede8b5ed-98ae-4b81-834f-62b908a6e51e'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:56:07.844926 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-ede8b5ed-98ae-4b81-834f-62b908a6e51e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7526d4f831844eae8549777bfb42639f', 'request_id': 'req-ede8b5ed-98ae-4b81-834f-62b908a6e51e'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:56:07.857354 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-ede8b5ed-98ae-4b81-834f-62b908a6e51e admin admin] The token cache is being invalidate because project 7526d4f831844eae8549777bfb42639f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:56:07.857640 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-ede8b5ed-98ae-4b81-834f-62b908a6e51e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:56:07.858277 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-ede8b5ed-98ae-4b81-834f-62b908a6e51e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:56:07.860058 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1128/2239] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:56:07 2022] DELETE /identity/v3/projects/7526d4f831844eae8549777bfb42639f => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:56:07.864829 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf32eefb-490a-48a9-81ad-adc7799d9f2f None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:07.865235 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf32eefb-490a-48a9-81ad-adc7799d9f2f None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:07.865383 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf32eefb-490a-48a9-81ad-adc7799d9f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:07.883411 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-cf32eefb-490a-48a9-81ad-adc7799d9f2f None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:56:07.884618 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-cf32eefb-490a-48a9-81ad-adc7799d9f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:56:07.944231 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1112/2240] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:56:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:56:08.010763 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a27936-52ae-47dd-b786-fc271e2742fe None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:08.011220 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a27936-52ae-47dd-b786-fc271e2742fe None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:08.011575 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a27936-52ae-47dd-b786-fc271e2742fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:08.031437 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-17a27936-52ae-47dd-b786-fc271e2742fe None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:56:08.035797 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-17a27936-52ae-47dd-b786-fc271e2742fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:56:08.040825 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1129/2241] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:56:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:56:08.045239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf32eefb-490a-48a9-81ad-adc7799d9f2f None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:56:08.055622 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf32eefb-490a-48a9-81ad-adc7799d9f2f None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:56:08.060809 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b787cd75-e59a-4e9f-9ea0-e4c7c28fa273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:56:08.061000 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b787cd75-e59a-4e9f-9ea0-e4c7c28fa273 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:08.061313 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b787cd75-e59a-4e9f-9ea0-e4c7c28fa273 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:08.061627 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b787cd75-e59a-4e9f-9ea0-e4c7c28fa273 admin admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:08.062431 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b787cd75-e59a-4e9f-9ea0-e4c7c28fa273 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:56:08.062743 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b787cd75-e59a-4e9f-9ea0-e4c7c28fa273 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:56:08.064426 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b787cd75-e59a-4e9f-9ea0-e4c7c28fa273 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:56:08.078701 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1113/2242] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 22:56:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:56:08.083329 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a27936-52ae-47dd-b786-fc271e2742fe None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:56:08.094701 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a27936-52ae-47dd-b786-fc271e2742fe None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:56:08.100202 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad87ea9-9d10-4a51-a98d-2458699cc595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:56:08.101216 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad87ea9-9d10-4a51-a98d-2458699cc595 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:08.101216 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad87ea9-9d10-4a51-a98d-2458699cc595 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:08.101448 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad87ea9-9d10-4a51-a98d-2458699cc595 admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:08.102318 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad87ea9-9d10-4a51-a98d-2458699cc595 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:56:08.104267 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad87ea9-9d10-4a51-a98d-2458699cc595 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:56:08.215439 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1130/2243] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:56:08 2022] POST /identity/v3/projects => generated 375 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:56:08.216649 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b787cd75-e59a-4e9f-9ea0-e4c7c28fa273 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:56:08.227598 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b787cd75-e59a-4e9f-9ea0-e4c7c28fa273 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:56:08.271001 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:56:08.289387 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf88895-276e-41b2-8ecb-864002c826d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:56:08.290088 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf88895-276e-41b2-8ecb-864002c826d1 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:08.290371 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf88895-276e-41b2-8ecb-864002c826d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:08.290646 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf88895-276e-41b2-8ecb-864002c826d1 admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:08.291362 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf88895-276e-41b2-8ecb-864002c826d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:56:08.293321 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf88895-276e-41b2-8ecb-864002c826d1 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:56:08.371114 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1114/2244] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:56:08 2022] POST /identity/v3/users => generated 288 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:56:08.375242 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad87ea9-9d10-4a51-a98d-2458699cc595 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:56:08.388058 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad87ea9-9d10-4a51-a98d-2458699cc595 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:56:08.393224 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93f7c7-63a4-4f65-b546-0da8cc8ad833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:56:08.394518 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93f7c7-63a4-4f65-b546-0da8cc8ad833 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:08.394518 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93f7c7-63a4-4f65-b546-0da8cc8ad833 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:08.394720 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93f7c7-63a4-4f65-b546-0da8cc8ad833 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:08.395384 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d93f7c7-63a4-4f65-b546-0da8cc8ad833 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:56:08.396981 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d93f7c7-63a4-4f65-b546-0da8cc8ad833 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:56:08.407761 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1131/2245] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:56:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:56:08.411963 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf88895-276e-41b2-8ecb-864002c826d1 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:56:08.421905 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf88895-276e-41b2-8ecb-864002c826d1 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:56:08.426702 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de690d5-9d84-4af0-8748-bf7b761e5c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:56:08.427454 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de690d5-9d84-4af0-8748-bf7b761e5c1f admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:08.427738 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de690d5-9d84-4af0-8748-bf7b761e5c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:08.428017 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de690d5-9d84-4af0-8748-bf7b761e5c1f admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:08.430864 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de690d5-9d84-4af0-8748-bf7b761e5c1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:56:08.430864 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de690d5-9d84-4af0-8748-bf7b761e5c1f admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:56:08.441525 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1115/2246] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:56:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:56:08.446386 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93f7c7-63a4-4f65-b546-0da8cc8ad833 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:56:08.459985 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93f7c7-63a4-4f65-b546-0da8cc8ad833 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:56:08.463691 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b323c4-e136-491c-8912-cb3afc6252ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:56:08.464606 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b323c4-e136-491c-8912-cb3afc6252ac admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:08.464889 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b323c4-e136-491c-8912-cb3afc6252ac admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:08.465164 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b323c4-e136-491c-8912-cb3afc6252ac admin admin] PATH_INFO: `/v3/projects/f64f9241e0da4e8f98fb1ad5b46d836d/users/8677c914de1d4eb1b3e8c36eabbb0ade/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:08.481236 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b323c4-e136-491c-8912-cb3afc6252ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=f64f9241e0da4e8f98fb1ad5b46d836d, user_id=8677c914de1d4eb1b3e8c36eabbb0ade, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:56:08.483925 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b323c4-e136-491c-8912-cb3afc6252ac admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:56:08.548464 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1132/2247] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:56:08 2022] PUT /identity/v3/projects/f64f9241e0da4e8f98fb1ad5b46d836d/users/8677c914de1d4eb1b3e8c36eabbb0ade/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:56:08.560115 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b2055-666a-4dc7-96b1-a0578f6b705f None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:08.570637 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b2055-666a-4dc7-96b1-a0578f6b705f None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:08.571177 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b2055-666a-4dc7-96b1-a0578f6b705f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:08.601214 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-1a6b2055-666a-4dc7-96b1-a0578f6b705f None None] MFA Rules not processed for user `8677c914de1d4eb1b3e8c36eabbb0ade`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:56:08.602480 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-1a6b2055-666a-4dc7-96b1-a0578f6b705f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:56:08.701070 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1116/2248] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:56:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:56:08.715467 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b323c4-e136-491c-8912-cb3afc6252ac admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:56:08.717388 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-c3b323c4-e136-491c-8912-cb3afc6252ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:56:08.807254 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b323c4-e136-491c-8912-cb3afc6252ac admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:56:08.812320 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa787d20-bbd5-41c9-b004-67991a67430c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:56:08.813157 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa787d20-bbd5-41c9-b004-67991a67430c service cinder] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:08.813441 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa787d20-bbd5-41c9-b004-67991a67430c service cinder] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:08.817873 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa787d20-bbd5-41c9-b004-67991a67430c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:08.823015 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa787d20-bbd5-41c9-b004-67991a67430c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:56:08.824463 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa787d20-bbd5-41c9-b004-67991a67430c service cinder] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:56:08.833878 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1133/2249] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:56:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:56:10.488607 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2b8743-3507-4eb9-9c17-7c4b332dfb01 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:10.489669 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2b8743-3507-4eb9-9c17-7c4b332dfb01 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:10.490423 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2b8743-3507-4eb9-9c17-7c4b332dfb01 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:10.493390 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1117/2250] 10.222.0.52 () {54 vars in 921 bytes} [Thu Jul 21 22:56:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:56:10.504865 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093690cf-6630-4d66-b8d3-ddac774b029d None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:10.505374 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093690cf-6630-4d66-b8d3-ddac774b029d None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:10.505660 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093690cf-6630-4d66-b8d3-ddac774b029d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:10.532196 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-093690cf-6630-4d66-b8d3-ddac774b029d None None] MFA Rules not processed for user `6bfdb655a2a44b33ab724daaf73a294b`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:56:10.533354 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-093690cf-6630-4d66-b8d3-ddac774b029d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:56:10.633736 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:56:10.655552 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1134/2251] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Jul 21 22:56:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:56:10.668653 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2b8743-3507-4eb9-9c17-7c4b332dfb01 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:56:10.670955 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-7c2b8743-3507-4eb9-9c17-7c4b332dfb01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:56:10.712978 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4dfacb-aa87-429e-ad06-1312d6fab98d None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:10.713274 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4dfacb-aa87-429e-ad06-1312d6fab98d None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:10.713559 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4dfacb-aa87-429e-ad06-1312d6fab98d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:10.733169 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-5a4dfacb-aa87-429e-ad06-1312d6fab98d None None] MFA Rules not processed for user `8677c914de1d4eb1b3e8c36eabbb0ade`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:56:10.734754 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-5a4dfacb-aa87-429e-ad06-1312d6fab98d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:56:10.742168 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2b8743-3507-4eb9-9c17-7c4b332dfb01 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:56:10.746251 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a9de67-3aa7-4f03-8b09-7df908fa80be service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '927778d005fd48a4b4b8fd3c4419bb36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:56:10.747151 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a9de67-3aa7-4f03-8b09-7df908fa80be service glance] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:10.747588 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a9de67-3aa7-4f03-8b09-7df908fa80be service glance] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:10.747898 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a9de67-3aa7-4f03-8b09-7df908fa80be service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:10.753518 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1135/2252] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:56:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:56:10.761056 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a9de67-3aa7-4f03-8b09-7df908fa80be service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:56:10.763284 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a9de67-3aa7-4f03-8b09-7df908fa80be service glance] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:56:10.778073 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1118/2253] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:56:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:56:11.010159 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e1b4e-870d-40d6-97aa-3152ff1be02d None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:11.010444 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e1b4e-870d-40d6-97aa-3152ff1be02d None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:11.010720 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e1b4e-870d-40d6-97aa-3152ff1be02d None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:11.012538 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1136/2254] 10.222.0.52 () {54 vars in 921 bytes} [Thu Jul 21 22:56:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:56:11.017465 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f3117-a10d-429a-8cfd-1ea171b30403 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:11.017768 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f3117-a10d-429a-8cfd-1ea171b30403 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:11.018045 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f3117-a10d-429a-8cfd-1ea171b30403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:11.036007 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-e20f3117-a10d-429a-8cfd-1ea171b30403 None None] MFA Rules not processed for user `48c3e33f24ff4d66a72f06848d769ac7`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:56:11.037709 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-e20f3117-a10d-429a-8cfd-1ea171b30403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:56:11.093709 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:56:11.100772 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1119/2255] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Jul 21 22:56:11 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:56:11.113240 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84e1b4e-870d-40d6-97aa-3152ff1be02d None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:56:11.115079 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-e84e1b4e-870d-40d6-97aa-3152ff1be02d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:56:11.129674 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84e1b4e-870d-40d6-97aa-3152ff1be02d None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:56:11.134134 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3789b411-60fd-4728-9ceb-c91c0fbd3460 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:56:11.135011 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3789b411-60fd-4728-9ceb-c91c0fbd3460 service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:11.135380 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3789b411-60fd-4728-9ceb-c91c0fbd3460 service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:11.135774 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3789b411-60fd-4728-9ceb-c91c0fbd3460 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:11.144424 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3789b411-60fd-4728-9ceb-c91c0fbd3460 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:56:11.145859 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3789b411-60fd-4728-9ceb-c91c0fbd3460 service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:56:11.154971 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1137/2256] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:56:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:56:11.276219 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60080bcf-4956-49b3-8004-005a3673cd20 None None] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:11.276831 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60080bcf-4956-49b3-8004-005a3673cd20 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:11.289613 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60080bcf-4956-49b3-8004-005a3673cd20 None None] PATH_INFO: `/` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:11.289613 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1120/2257] 10.222.0.52 () {54 vars in 921 bytes} [Thu Jul 21 22:56:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 22:56:11.295100 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9477a821-039f-4f25-a1b5-65dc60454126 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:11.295391 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9477a821-039f-4f25-a1b5-65dc60454126 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:11.295668 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9477a821-039f-4f25-a1b5-65dc60454126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:11.315257 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-9477a821-039f-4f25-a1b5-65dc60454126 None None] MFA Rules not processed for user `6bfdb655a2a44b33ab724daaf73a294b`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:56:11.316452 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-9477a821-039f-4f25-a1b5-65dc60454126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:56:11.327572 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1138/2258] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Jul 21 22:56:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:56:11.338992 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60080bcf-4956-49b3-8004-005a3673cd20 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:56:11.350954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60080bcf-4956-49b3-8004-005a3673cd20 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:56:11.355379 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1020a0cb-2f39-428b-95e7-ef7ebdaaae64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:56:11.356197 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020a0cb-2f39-428b-95e7-ef7ebdaaae64 service neutron] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:11.356607 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020a0cb-2f39-428b-95e7-ef7ebdaaae64 service neutron] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:11.356918 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020a0cb-2f39-428b-95e7-ef7ebdaaae64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:11.368502 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1020a0cb-2f39-428b-95e7-ef7ebdaaae64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:56:11.371586 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1020a0cb-2f39-428b-95e7-ef7ebdaaae64 service neutron] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:56:11.383301 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1121/2259] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:56:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:56:18.462456 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9477a821-039f-4f25-a1b5-65dc60454126 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:56:18.464741 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-9477a821-039f-4f25-a1b5-65dc60454126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:56:18.569371 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9477a821-039f-4f25-a1b5-65dc60454126 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:56:18.574472 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33df918d-e85b-4d0b-93e4-b5b6476aaa4c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10c0bed169344a77bfad29624c5e5a8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:56:18.575229 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33df918d-e85b-4d0b-93e4-b5b6476aaa4c service placement] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:18.575576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33df918d-e85b-4d0b-93e4-b5b6476aaa4c service placement] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:18.575852 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33df918d-e85b-4d0b-93e4-b5b6476aaa4c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:18.577782 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-33df918d-e85b-4d0b-93e4-b5b6476aaa4c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:56:18.589483 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33df918d-e85b-4d0b-93e4-b5b6476aaa4c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:56:18.591081 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33df918d-e85b-4d0b-93e4-b5b6476aaa4c service placement] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:56:18.601304 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1139/2260] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:56:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:56:21.378276 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=6 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:56:27.785471 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7be8b0-54d8-4e99-ab78-92c8f5f3a2bc None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:27.786058 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7be8b0-54d8-4e99-ab78-92c8f5f3a2bc None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:27.786155 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7be8b0-54d8-4e99-ab78-92c8f5f3a2bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:27.805107 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-9b7be8b0-54d8-4e99-ab78-92c8f5f3a2bc None None] MFA Rules not processed for user `63cc904578554aea84ff1917a5ed075e`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:56:27.806326 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-9b7be8b0-54d8-4e99-ab78-92c8f5f3a2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:56:27.866971 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1122/2261] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jul 21 22:56:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:56:27.878005 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33df918d-e85b-4d0b-93e4-b5b6476aaa4c service placement] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:56:27.880237 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-33df918d-e85b-4d0b-93e4-b5b6476aaa4c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:56:27.950086 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33df918d-e85b-4d0b-93e4-b5b6476aaa4c service placement] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:56:27.955263 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8c432c-f12e-4bc1-803c-6ded8af7cfc0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba7eef4e1b27476ab4016449d9c3a4b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:56:27.955982 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8c432c-f12e-4bc1-803c-6ded8af7cfc0 service swift] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:27.956269 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8c432c-f12e-4bc1-803c-6ded8af7cfc0 service swift] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:27.956549 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8c432c-f12e-4bc1-803c-6ded8af7cfc0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:27.965147 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8c432c-f12e-4bc1-803c-6ded8af7cfc0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:56:27.967611 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8c432c-f12e-4bc1-803c-6ded8af7cfc0 service swift] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:56:27.978946 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1140/2262] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jul 21 22:56:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 22:56:37.856781 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=25 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:56:37.974060 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=82 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:56:52.741157 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febd6fe-7505-4bb3-a9fd-5582e0e8d5e0 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:52.741670 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febd6fe-7505-4bb3-a9fd-5582e0e8d5e0 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:52.741768 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febd6fe-7505-4bb3-a9fd-5582e0e8d5e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:52.759383 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-6febd6fe-7505-4bb3-a9fd-5582e0e8d5e0 None None] MFA Rules not processed for user `63cc904578554aea84ff1917a5ed075e`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:56:52.760570 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-6febd6fe-7505-4bb3-a9fd-5582e0e8d5e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:56:52.768306 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1123/2263] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jul 21 22:56:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:56:52.789974 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8c432c-f12e-4bc1-803c-6ded8af7cfc0 service swift] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:56:52.800415 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8c432c-f12e-4bc1-803c-6ded8af7cfc0 service swift] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:56:52.805480 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd75ae8-f7c1-44a5-a9d6-575ecae70d73 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba7eef4e1b27476ab4016449d9c3a4b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:56:52.806116 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd75ae8-f7c1-44a5-a9d6-575ecae70d73 service swift] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:52.806402 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd75ae8-f7c1-44a5-a9d6-575ecae70d73 service swift] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:52.806679 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd75ae8-f7c1-44a5-a9d6-575ecae70d73 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:52.814947 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd75ae8-f7c1-44a5-a9d6-575ecae70d73 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:56:52.816534 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd75ae8-f7c1-44a5-a9d6-575ecae70d73 service swift] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:56:52.826116 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1141/2264] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jul 21 22:56:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 22:56:53.626369 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e7750b-28f3-4cac-9302-3ffe8ef97d3c None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:53.626654 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e7750b-28f3-4cac-9302-3ffe8ef97d3c None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:53.627202 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e7750b-28f3-4cac-9302-3ffe8ef97d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:53.646132 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-c1e7750b-28f3-4cac-9302-3ffe8ef97d3c None None] MFA Rules not processed for user `63cc904578554aea84ff1917a5ed075e`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:56:53.647387 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-c1e7750b-28f3-4cac-9302-3ffe8ef97d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:56:53.658543 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1124/2265] 10.222.0.52 () {58 vars in 967 bytes} [Thu Jul 21 22:56:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:56:53.668860 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd75ae8-f7c1-44a5-a9d6-575ecae70d73 service swift] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:56:53.679662 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd75ae8-f7c1-44a5-a9d6-575ecae70d73 service swift] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:56:53.689402 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998423c6-9f21-4d7d-8ec7-601a964bfa61 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba7eef4e1b27476ab4016449d9c3a4b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:56:53.690315 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998423c6-9f21-4d7d-8ec7-601a964bfa61 service swift] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:53.690760 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998423c6-9f21-4d7d-8ec7-601a964bfa61 service swift] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:53.691283 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998423c6-9f21-4d7d-8ec7-601a964bfa61 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:53.700369 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998423c6-9f21-4d7d-8ec7-601a964bfa61 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:56:53.702919 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998423c6-9f21-4d7d-8ec7-601a964bfa61 service swift] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:56:53.712770 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1142/2266] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jul 21 22:56:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 22:56:54.286207 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6577b0-2190-4b2d-98e8-671c9b9a0514 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:54.286706 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6577b0-2190-4b2d-98e8-671c9b9a0514 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:54.286855 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6577b0-2190-4b2d-98e8-671c9b9a0514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:54.305126 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-2a6577b0-2190-4b2d-98e8-671c9b9a0514 None None] MFA Rules not processed for user `63cc904578554aea84ff1917a5ed075e`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:56:54.306331 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-2a6577b0-2190-4b2d-98e8-671c9b9a0514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:56:54.313511 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1125/2267] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jul 21 22:56:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:56:54.325458 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998423c6-9f21-4d7d-8ec7-601a964bfa61 service swift] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:56:54.336592 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998423c6-9f21-4d7d-8ec7-601a964bfa61 service swift] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:56:54.342005 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce808d-ea5f-45cb-ab7f-f4e43a78f9ad service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba7eef4e1b27476ab4016449d9c3a4b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:56:54.344998 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce808d-ea5f-45cb-ab7f-f4e43a78f9ad service swift] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:54.344998 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce808d-ea5f-45cb-ab7f-f4e43a78f9ad service swift] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:54.344998 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce808d-ea5f-45cb-ab7f-f4e43a78f9ad service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:54.352306 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfce808d-ea5f-45cb-ab7f-f4e43a78f9ad service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:56:54.353856 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfce808d-ea5f-45cb-ab7f-f4e43a78f9ad service swift] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:56:54.361794 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1143/2268] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jul 21 22:56:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 22:56:54.965976 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01d967-8f52-4089-82c5-3eaec7aa62a0 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:54.966348 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01d967-8f52-4089-82c5-3eaec7aa62a0 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:54.966664 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01d967-8f52-4089-82c5-3eaec7aa62a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:54.986259 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-6e01d967-8f52-4089-82c5-3eaec7aa62a0 None None] MFA Rules not processed for user `63cc904578554aea84ff1917a5ed075e`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:56:54.987521 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-6e01d967-8f52-4089-82c5-3eaec7aa62a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:56:54.995026 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1126/2269] 10.222.0.52 () {58 vars in 967 bytes} [Thu Jul 21 22:56:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:56:55.007993 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce808d-ea5f-45cb-ab7f-f4e43a78f9ad service swift] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:56:55.018754 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce808d-ea5f-45cb-ab7f-f4e43a78f9ad service swift] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:56:55.024170 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d4329b-6f0b-464d-8617-f60009d832c1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba7eef4e1b27476ab4016449d9c3a4b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:56:55.024954 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4329b-6f0b-464d-8617-f60009d832c1 service swift] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:55.025339 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4329b-6f0b-464d-8617-f60009d832c1 service swift] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:55.025687 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4329b-6f0b-464d-8617-f60009d832c1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:55.034808 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d4329b-6f0b-464d-8617-f60009d832c1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:56:55.036506 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d4329b-6f0b-464d-8617-f60009d832c1 service swift] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:56:55.052253 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1144/2270] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jul 21 22:56:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 22:56:56.127801 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df74fdee-8175-4dca-80cc-470930c71242 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:56.128307 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df74fdee-8175-4dca-80cc-470930c71242 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:56.128412 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df74fdee-8175-4dca-80cc-470930c71242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:56.146044 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-df74fdee-8175-4dca-80cc-470930c71242 None None] MFA Rules not processed for user `8c11dc2f4c764dc58458d9226efc878e`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:56:56.147745 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-df74fdee-8175-4dca-80cc-470930c71242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:56:56.202070 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1127/2271] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:56:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:56:57.556792 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6e8f5-a28b-43a5-aaac-d4bf5449cfba None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:57.557243 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6e8f5-a28b-43a5-aaac-d4bf5449cfba None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:57.557386 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6e8f5-a28b-43a5-aaac-d4bf5449cfba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:57.576318 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-d6f6e8f5-a28b-43a5-aaac-d4bf5449cfba None None] MFA Rules not processed for user `63cc904578554aea84ff1917a5ed075e`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:56:57.577565 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-d6f6e8f5-a28b-43a5-aaac-d4bf5449cfba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:56:57.585289 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1145/2272] 10.222.0.52 () {58 vars in 967 bytes} [Thu Jul 21 22:56:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:56:57.601888 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df74fdee-8175-4dca-80cc-470930c71242 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:56:57.614408 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df74fdee-8175-4dca-80cc-470930c71242 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:56:57.619295 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c41745f-6672-4321-abb4-850b65752e93 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba7eef4e1b27476ab4016449d9c3a4b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:56:57.620252 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41745f-6672-4321-abb4-850b65752e93 service swift] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:57.620252 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41745f-6672-4321-abb4-850b65752e93 service swift] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:57.620617 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41745f-6672-4321-abb4-850b65752e93 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:57.628658 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c41745f-6672-4321-abb4-850b65752e93 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:56:57.630224 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c41745f-6672-4321-abb4-850b65752e93 service swift] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:56:57.638238 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1128/2273] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jul 21 22:56:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 22:56:57.725168 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f6e8f5-a28b-43a5-aaac-d4bf5449cfba None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:56:57.792973 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f6e8f5-a28b-43a5-aaac-d4bf5449cfba None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:56:57.799648 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3009c2-b9d1-47de-b6ec-b32931f91bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:56:57.800586 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3009c2-b9d1-47de-b6ec-b32931f91bfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:57.800867 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3009c2-b9d1-47de-b6ec-b32931f91bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:57.801140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3009c2-b9d1-47de-b6ec-b32931f91bfd admin admin] PATH_INFO: `/v3/users/8677c914de1d4eb1b3e8c36eabbb0ade` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:57.802440 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3009c2-b9d1-47de-b6ec-b32931f91bfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=8677c914de1d4eb1b3e8c36eabbb0ade)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:56:57.804392 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3009c2-b9d1-47de-b6ec-b32931f91bfd admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:56:58.018761 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-0c3009c2-b9d1-47de-b6ec-b32931f91bfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8677c914de1d4eb1b3e8c36eabbb0ade'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:56:58.025673 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-0c3009c2-b9d1-47de-b6ec-b32931f91bfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8677c914de1d4eb1b3e8c36eabbb0ade'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:56:58.026578 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-0c3009c2-b9d1-47de-b6ec-b32931f91bfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8677c914de1d4eb1b3e8c36eabbb0ade'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:56:58.040934 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-0c3009c2-b9d1-47de-b6ec-b32931f91bfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8677c914de1d4eb1b3e8c36eabbb0ade'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:56:58.131933 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-0c3009c2-b9d1-47de-b6ec-b32931f91bfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8677c914de1d4eb1b3e8c36eabbb0ade'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:56:58.138579 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1146/2274] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:56:57 2022] DELETE /identity/v3/users/8677c914de1d4eb1b3e8c36eabbb0ade => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:56:58.143894 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c24bc5-8ae5-4dec-9d3a-88e5db4d58cd None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:58.144187 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c24bc5-8ae5-4dec-9d3a-88e5db4d58cd None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:58.144463 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c24bc5-8ae5-4dec-9d3a-88e5db4d58cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:58.162428 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-04c24bc5-8ae5-4dec-9d3a-88e5db4d58cd None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:56:58.164503 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-04c24bc5-8ae5-4dec-9d3a-88e5db4d58cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:56:58.229917 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1129/2275] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:56:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:56:58.241309 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3009c2-b9d1-47de-b6ec-b32931f91bfd admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:56:58.243306 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-0c3009c2-b9d1-47de-b6ec-b32931f91bfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:56:58.270402 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:56:58.310222 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3009c2-b9d1-47de-b6ec-b32931f91bfd admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:56:58.313998 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8b6a0b-7f8a-4199-9023-fd7832f6f842 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:56:58.314646 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b6a0b-7f8a-4199-9023-fd7832f6f842 service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:58.314973 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b6a0b-7f8a-4199-9023-fd7832f6f842 service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:58.315250 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b6a0b-7f8a-4199-9023-fd7832f6f842 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:58.324289 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8b6a0b-7f8a-4199-9023-fd7832f6f842 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:56:58.325783 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8b6a0b-7f8a-4199-9023-fd7832f6f842 service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:56:58.335948 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1147/2276] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:56:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:56:58.601791 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c24bc5-8ae5-4dec-9d3a-88e5db4d58cd None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:56:58.603414 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-04c24bc5-8ae5-4dec-9d3a-88e5db4d58cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:56:58.615495 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c24bc5-8ae5-4dec-9d3a-88e5db4d58cd None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:56:58.621544 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da7ac1-4cf1-4c55-9897-e38e3268e72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:56:58.622416 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da7ac1-4cf1-4c55-9897-e38e3268e72a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:58.622709 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da7ac1-4cf1-4c55-9897-e38e3268e72a admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:58.623020 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da7ac1-4cf1-4c55-9897-e38e3268e72a admin admin] PATH_INFO: `/v3/projects/f64f9241e0da4e8f98fb1ad5b46d836d` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:58.624785 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26da7ac1-4cf1-4c55-9897-e38e3268e72a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f64f9241e0da4e8f98fb1ad5b46d836d)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:56:58.626618 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26da7ac1-4cf1-4c55-9897-e38e3268e72a admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:56:58.781430 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-26da7ac1-4cf1-4c55-9897-e38e3268e72a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f64f9241e0da4e8f98fb1ad5b46d836d', 'request_id': 'req-26da7ac1-4cf1-4c55-9897-e38e3268e72a'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:56:58.787702 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-26da7ac1-4cf1-4c55-9897-e38e3268e72a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f64f9241e0da4e8f98fb1ad5b46d836d', 'request_id': 'req-26da7ac1-4cf1-4c55-9897-e38e3268e72a'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:56:58.790983 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:56:58.875859 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-26da7ac1-4cf1-4c55-9897-e38e3268e72a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f64f9241e0da4e8f98fb1ad5b46d836d', 'request_id': 'req-26da7ac1-4cf1-4c55-9897-e38e3268e72a'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:56:58.888955 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-26da7ac1-4cf1-4c55-9897-e38e3268e72a admin admin] The token cache is being invalidate because project f64f9241e0da4e8f98fb1ad5b46d836d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:56:58.889181 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-26da7ac1-4cf1-4c55-9897-e38e3268e72a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:56:58.889848 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-26da7ac1-4cf1-4c55-9897-e38e3268e72a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:56:58.891474 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1130/2277] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:56:58 2022] DELETE /identity/v3/projects/f64f9241e0da4e8f98fb1ad5b46d836d => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:56:58.896575 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6bde8-68c3-45bd-a67d-d17e0bc9c05a None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:58.897032 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6bde8-68c3-45bd-a67d-d17e0bc9c05a None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:58.897158 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6bde8-68c3-45bd-a67d-d17e0bc9c05a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:58.915012 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-73d6bde8-68c3-45bd-a67d-d17e0bc9c05a None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:56:58.916198 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-73d6bde8-68c3-45bd-a67d-d17e0bc9c05a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:56:58.979011 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1148/2278] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:56:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:56:58.987542 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658c629-7538-45af-ae4f-8dd3d2847c2a None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:58.987835 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658c629-7538-45af-ae4f-8dd3d2847c2a None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:58.988121 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658c629-7538-45af-ae4f-8dd3d2847c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:59.007394 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-5658c629-7538-45af-ae4f-8dd3d2847c2a None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:56:59.008612 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-5658c629-7538-45af-ae4f-8dd3d2847c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:56:59.017358 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1131/2279] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:56:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:56:59.021586 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d6bde8-68c3-45bd-a67d-d17e0bc9c05a None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:56:59.031577 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d6bde8-68c3-45bd-a67d-d17e0bc9c05a None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:56:59.036127 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb815b2-1e28-4e3d-bd6a-f9ae44726297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:56:59.036809 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb815b2-1e28-4e3d-bd6a-f9ae44726297 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:59.037120 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb815b2-1e28-4e3d-bd6a-f9ae44726297 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:59.037392 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb815b2-1e28-4e3d-bd6a-f9ae44726297 admin admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:59.038117 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb815b2-1e28-4e3d-bd6a-f9ae44726297 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:56:59.038421 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb815b2-1e28-4e3d-bd6a-f9ae44726297 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:56:59.040072 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb815b2-1e28-4e3d-bd6a-f9ae44726297 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:56:59.052053 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1149/2280] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 22:56:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:56:59.056404 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5658c629-7538-45af-ae4f-8dd3d2847c2a None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:56:59.066167 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5658c629-7538-45af-ae4f-8dd3d2847c2a None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:56:59.070825 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dac7dd0-7a45-4c00-9508-4c22cf8b3a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:56:59.071479 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dac7dd0-7a45-4c00-9508-4c22cf8b3a30 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:59.071765 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dac7dd0-7a45-4c00-9508-4c22cf8b3a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:59.072042 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dac7dd0-7a45-4c00-9508-4c22cf8b3a30 admin admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:59.072741 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dac7dd0-7a45-4c00-9508-4c22cf8b3a30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:56:59.074633 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dac7dd0-7a45-4c00-9508-4c22cf8b3a30 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:56:59.188234 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1132/2281] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:56:59 2022] POST /identity/v3/projects => generated 383 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:56:59.189416 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb815b2-1e28-4e3d-bd6a-f9ae44726297 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:56:59.199885 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb815b2-1e28-4e3d-bd6a-f9ae44726297 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:56:59.267860 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082f81ac-4df0-4b45-ade9-c3976954a97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:56:59.268537 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f81ac-4df0-4b45-ade9-c3976954a97b admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:59.268820 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f81ac-4df0-4b45-ade9-c3976954a97b admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:59.269095 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f81ac-4df0-4b45-ade9-c3976954a97b admin admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:59.269812 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082f81ac-4df0-4b45-ade9-c3976954a97b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:56:59.271743 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082f81ac-4df0-4b45-ade9-c3976954a97b admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:56:59.389653 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1150/2282] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:56:59 2022] POST /identity/v3/users => generated 292 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:56:59.393771 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dac7dd0-7a45-4c00-9508-4c22cf8b3a30 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:56:59.404127 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dac7dd0-7a45-4c00-9508-4c22cf8b3a30 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:56:59.409209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1558ca78-58a6-4286-a9d3-43bebeb8b2c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:56:59.409806 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558ca78-58a6-4286-a9d3-43bebeb8b2c9 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:59.410090 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558ca78-58a6-4286-a9d3-43bebeb8b2c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:59.410365 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558ca78-58a6-4286-a9d3-43bebeb8b2c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:59.411125 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1558ca78-58a6-4286-a9d3-43bebeb8b2c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:56:59.412783 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1558ca78-58a6-4286-a9d3-43bebeb8b2c9 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:56:59.427185 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1133/2283] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:56:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:56:59.429247 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082f81ac-4df0-4b45-ade9-c3976954a97b admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:56:59.439543 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082f81ac-4df0-4b45-ade9-c3976954a97b admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:56:59.444312 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd7c11d-e2ae-467e-be58-c6cbf0905ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:56:59.446136 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd7c11d-e2ae-467e-be58-c6cbf0905ea7 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:59.446136 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd7c11d-e2ae-467e-be58-c6cbf0905ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:59.446136 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd7c11d-e2ae-467e-be58-c6cbf0905ea7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:59.446390 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd7c11d-e2ae-467e-be58-c6cbf0905ea7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:56:59.447796 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd7c11d-e2ae-467e-be58-c6cbf0905ea7 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:56:59.458347 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1151/2284] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:56:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:56:59.462507 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1558ca78-58a6-4286-a9d3-43bebeb8b2c9 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:56:59.473721 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1558ca78-58a6-4286-a9d3-43bebeb8b2c9 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:56:59.478524 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6727df1e-dd84-424c-ba8b-a2a4de28dbd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:56:59.479448 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727df1e-dd84-424c-ba8b-a2a4de28dbd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:59.479738 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727df1e-dd84-424c-ba8b-a2a4de28dbd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:59.480018 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727df1e-dd84-424c-ba8b-a2a4de28dbd9 admin admin] PATH_INFO: `/v3/projects/5dbf5236757941d6b4db3517e1757621/users/257adcc88884437eb740f78dbd9e19c2/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:59.494957 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6727df1e-dd84-424c-ba8b-a2a4de28dbd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dbf5236757941d6b4db3517e1757621, user_id=257adcc88884437eb740f78dbd9e19c2, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:56:59.497600 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6727df1e-dd84-424c-ba8b-a2a4de28dbd9 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:56:59.586572 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1134/2285] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:56:59 2022] PUT /identity/v3/projects/5dbf5236757941d6b4db3517e1757621/users/257adcc88884437eb740f78dbd9e19c2/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:56:59.591717 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953411f-0501-47f5-8564-530ac7d15b30 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:59.592005 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953411f-0501-47f5-8564-530ac7d15b30 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:59.592287 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953411f-0501-47f5-8564-530ac7d15b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:59.610654 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-7953411f-0501-47f5-8564-530ac7d15b30 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:56:59.611878 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-7953411f-0501-47f5-8564-530ac7d15b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:56:59.635811 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:56:59.671067 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1152/2286] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:56:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:56:59.680606 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6727df1e-dd84-424c-ba8b-a2a4de28dbd9 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:56:59.682344 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-6727df1e-dd84-424c-ba8b-a2a4de28dbd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:56:59.739333 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6727df1e-dd84-424c-ba8b-a2a4de28dbd9 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:56:59.742968 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165d487e-3530-49e3-b89e-0e6a4292bcdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:56:59.743685 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d487e-3530-49e3-b89e-0e6a4292bcdf service neutron] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:56:59.743976 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d487e-3530-49e3-b89e-0e6a4292bcdf service neutron] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:56:59.744252 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d487e-3530-49e3-b89e-0e6a4292bcdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:56:59.753095 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165d487e-3530-49e3-b89e-0e6a4292bcdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:56:59.754509 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165d487e-3530-49e3-b89e-0e6a4292bcdf service neutron] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:56:59.764005 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1135/2287] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:56:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:57:02.579943 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac1264-e783-4593-a474-9e3600493021 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:57:02.580409 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac1264-e783-4593-a474-9e3600493021 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:57:02.580503 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac1264-e783-4593-a474-9e3600493021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:57:02.601367 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-f8ac1264-e783-4593-a474-9e3600493021 None None] MFA Rules not processed for user `5eadb243fd6a4517bca72e18dec30e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:57:02.602826 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-f8ac1264-e783-4593-a474-9e3600493021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:57:02.721797 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1153/2288] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:57:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:57:05.715805 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c1175c-8d44-474e-a12b-7019694e9ea9 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:57:05.716331 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c1175c-8d44-474e-a12b-7019694e9ea9 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:57:05.716453 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c1175c-8d44-474e-a12b-7019694e9ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:57:05.746654 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-b7c1175c-8d44-474e-a12b-7019694e9ea9 None None] MFA Rules not processed for user `257adcc88884437eb740f78dbd9e19c2`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:57:05.747966 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-b7c1175c-8d44-474e-a12b-7019694e9ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:57:05.823114 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1136/2289] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:57:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:57:05.827587 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ac1264-e783-4593-a474-9e3600493021 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:57:05.840447 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ac1264-e783-4593-a474-9e3600493021 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:57:05.845732 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a1509-a674-4220-ac30-2d11b9b47366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:57:05.846854 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a1509-a674-4220-ac30-2d11b9b47366 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:57:05.847144 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a1509-a674-4220-ac30-2d11b9b47366 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:57:05.847423 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a1509-a674-4220-ac30-2d11b9b47366 admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:57:05.848136 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62a1509-a674-4220-ac30-2d11b9b47366 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:57:05.850057 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62a1509-a674-4220-ac30-2d11b9b47366 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:57:05.954804 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1154/2290] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:57:05 2022] POST /identity/v3/projects => generated 385 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:57:05.956301 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c1175c-8d44-474e-a12b-7019694e9ea9 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:57:05.967503 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c1175c-8d44-474e-a12b-7019694e9ea9 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:57:05.981080 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:57:06.031601 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf93f7-71e0-4cc7-896c-b1025de3dbeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:57:06.032291 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf93f7-71e0-4cc7-896c-b1025de3dbeb admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:57:06.032575 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf93f7-71e0-4cc7-896c-b1025de3dbeb admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:57:06.032856 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf93f7-71e0-4cc7-896c-b1025de3dbeb admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:57:06.033579 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cf93f7-71e0-4cc7-896c-b1025de3dbeb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:57:06.041225 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cf93f7-71e0-4cc7-896c-b1025de3dbeb admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:57:06.139733 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1137/2291] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:57:05 2022] POST /identity/v3/users => generated 299 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:57:06.143587 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a1509-a674-4220-ac30-2d11b9b47366 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:57:06.157161 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a1509-a674-4220-ac30-2d11b9b47366 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:57:06.162201 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1a0dec-c4f2-4060-bbd0-93b1f80b0e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:57:06.163007 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a0dec-c4f2-4060-bbd0-93b1f80b0e34 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:57:06.163295 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a0dec-c4f2-4060-bbd0-93b1f80b0e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:57:06.163578 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a0dec-c4f2-4060-bbd0-93b1f80b0e34 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:57:06.164315 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1a0dec-c4f2-4060-bbd0-93b1f80b0e34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:57:06.165913 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1a0dec-c4f2-4060-bbd0-93b1f80b0e34 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:57:06.177955 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1155/2292] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:57:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:57:06.182239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf93f7-71e0-4cc7-896c-b1025de3dbeb admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:57:06.193055 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf93f7-71e0-4cc7-896c-b1025de3dbeb admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:57:06.198262 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823e8e30-aae9-4d8a-ba01-1a21bb709a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:57:06.199111 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823e8e30-aae9-4d8a-ba01-1a21bb709a05 admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:57:06.199394 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823e8e30-aae9-4d8a-ba01-1a21bb709a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:57:06.199674 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823e8e30-aae9-4d8a-ba01-1a21bb709a05 admin admin] PATH_INFO: `/v3/projects/6a7d54c1b920447f9bab1caf2e579f79/users/4d7b543f09474961938ddb1585bd0a38/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:57:06.216718 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823e8e30-aae9-4d8a-ba01-1a21bb709a05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a7d54c1b920447f9bab1caf2e579f79, user_id=4d7b543f09474961938ddb1585bd0a38, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:57:06.220138 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823e8e30-aae9-4d8a-ba01-1a21bb709a05 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:57:06.293578 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1138/2293] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:57:06 2022] PUT /identity/v3/projects/6a7d54c1b920447f9bab1caf2e579f79/users/4d7b543f09474961938ddb1585bd0a38/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:57:06.297624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1a0dec-c4f2-4060-bbd0-93b1f80b0e34 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:57:06.313094 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1a0dec-c4f2-4060-bbd0-93b1f80b0e34 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:57:06.382604 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f6a052-7e9e-470b-8502-0c8b7d281151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:57:06.383334 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f6a052-7e9e-470b-8502-0c8b7d281151 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:57:06.383617 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f6a052-7e9e-470b-8502-0c8b7d281151 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:57:06.383893 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f6a052-7e9e-470b-8502-0c8b7d281151 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:57:06.384635 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f6a052-7e9e-470b-8502-0c8b7d281151 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:57:06.386242 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f6a052-7e9e-470b-8502-0c8b7d281151 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:57:06.398724 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1156/2294] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:57:06 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:57:06.403315 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823e8e30-aae9-4d8a-ba01-1a21bb709a05 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:57:06.414269 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823e8e30-aae9-4d8a-ba01-1a21bb709a05 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:57:06.419615 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b9d4f5-9fcc-4e3f-be1c-93c7e070dbf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:57:06.420515 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9d4f5-9fcc-4e3f-be1c-93c7e070dbf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:57:06.420928 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9d4f5-9fcc-4e3f-be1c-93c7e070dbf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:57:06.421209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9d4f5-9fcc-4e3f-be1c-93c7e070dbf7 admin admin] PATH_INFO: `/v3/projects/6a7d54c1b920447f9bab1caf2e579f79/users/4d7b543f09474961938ddb1585bd0a38/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:57:06.423268 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b9d4f5-9fcc-4e3f-be1c-93c7e070dbf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a7d54c1b920447f9bab1caf2e579f79, user_id=4d7b543f09474961938ddb1585bd0a38, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:57:06.426907 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b9d4f5-9fcc-4e3f-be1c-93c7e070dbf7 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:57:06.439260 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1139/2295] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:57:06 2022] PUT /identity/v3/projects/6a7d54c1b920447f9bab1caf2e579f79/users/4d7b543f09474961938ddb1585bd0a38/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:57:13.792143 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a9657-d1d2-433e-b995-d295dbf33ac2 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:57:13.792657 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a9657-d1d2-433e-b995-d295dbf33ac2 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:57:13.792909 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a9657-d1d2-433e-b995-d295dbf33ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:57:13.824003 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-2c4a9657-d1d2-433e-b995-d295dbf33ac2 None None] MFA Rules not processed for user `4d7b543f09474961938ddb1585bd0a38`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:57:13.825241 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-2c4a9657-d1d2-433e-b995-d295dbf33ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:57:13.853394 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:57:13.895577 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1157/2296] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:57:13 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:57:13.908289 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b9d4f5-9fcc-4e3f-be1c-93c7e070dbf7 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:57:13.910119 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-72b9d4f5-9fcc-4e3f-be1c-93c7e070dbf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:57:13.972907 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b9d4f5-9fcc-4e3f-be1c-93c7e070dbf7 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:57:13.976938 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d5bef0-7773-47e1-af4a-94ab8583552a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '927778d005fd48a4b4b8fd3c4419bb36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:57:13.977618 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5bef0-7773-47e1-af4a-94ab8583552a service glance] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:57:13.977939 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5bef0-7773-47e1-af4a-94ab8583552a service glance] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:57:13.978215 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5bef0-7773-47e1-af4a-94ab8583552a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:57:14.060435 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d5bef0-7773-47e1-af4a-94ab8583552a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:57:14.061926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d5bef0-7773-47e1-af4a-94ab8583552a service glance] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:57:14.080896 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1140/2297] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:57:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:57:14.396410 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e3d30f-7fae-4e23-817a-2e90f7a0f8de None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:57:14.399433 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e3d30f-7fae-4e23-817a-2e90f7a0f8de None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:57:14.399433 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e3d30f-7fae-4e23-817a-2e90f7a0f8de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:57:14.419098 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-15e3d30f-7fae-4e23-817a-2e90f7a0f8de None None] MFA Rules not processed for user `63cc904578554aea84ff1917a5ed075e`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:57:14.420478 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-15e3d30f-7fae-4e23-817a-2e90f7a0f8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:57:14.502945 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1158/2298] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jul 21 22:57:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:57:14.511888 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d5bef0-7773-47e1-af4a-94ab8583552a service glance] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:57:14.513221 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-f2d5bef0-7773-47e1-af4a-94ab8583552a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:57:14.549638 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:57:14.595868 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d5bef0-7773-47e1-af4a-94ab8583552a service glance] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:57:14.613931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77090b7c-1b6a-4fcb-9653-3e3f3658e5c4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba7eef4e1b27476ab4016449d9c3a4b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:57:14.614243 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77090b7c-1b6a-4fcb-9653-3e3f3658e5c4 service swift] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:57:14.614505 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77090b7c-1b6a-4fcb-9653-3e3f3658e5c4 service swift] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:57:14.614820 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77090b7c-1b6a-4fcb-9653-3e3f3658e5c4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:57:14.624545 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77090b7c-1b6a-4fcb-9653-3e3f3658e5c4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:57:14.626539 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77090b7c-1b6a-4fcb-9653-3e3f3658e5c4 service swift] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:57:14.636349 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1141/2299] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jul 21 22:57:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 22:57:24.490032 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=42 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:57:25.079213 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=21 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:57:26.443068 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5856980-083c-4a3b-9064-c5204d92ec8a None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:57:26.443582 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5856980-083c-4a3b-9064-c5204d92ec8a None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:57:26.443682 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5856980-083c-4a3b-9064-c5204d92ec8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:57:26.461685 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-e5856980-083c-4a3b-9064-c5204d92ec8a None None] MFA Rules not processed for user `257adcc88884437eb740f78dbd9e19c2`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:57:26.462914 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-e5856980-083c-4a3b-9064-c5204d92ec8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:57:26.470456 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1159/2300] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:57:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:57:30.765496 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ce644d-66c9-4ba5-a9e3-f1ee7216b571 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:57:30.765947 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ce644d-66c9-4ba5-a9e3-f1ee7216b571 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:57:30.766066 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ce644d-66c9-4ba5-a9e3-f1ee7216b571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:57:30.785941 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-58ce644d-66c9-4ba5-a9e3-f1ee7216b571 None None] MFA Rules not processed for user `63cc904578554aea84ff1917a5ed075e`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:57:30.787235 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-58ce644d-66c9-4ba5-a9e3-f1ee7216b571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:57:30.810004 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1142/2301] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jul 21 22:57:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:57:30.827236 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5856980-083c-4a3b-9064-c5204d92ec8a None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:57:30.843438 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5856980-083c-4a3b-9064-c5204d92ec8a None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:57:30.853324 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed70923-a988-4d91-ae36-9bb927ff97f9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba7eef4e1b27476ab4016449d9c3a4b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:57:30.854028 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed70923-a988-4d91-ae36-9bb927ff97f9 service swift] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:57:30.854528 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed70923-a988-4d91-ae36-9bb927ff97f9 service swift] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:57:30.855134 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed70923-a988-4d91-ae36-9bb927ff97f9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:57:30.864350 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed70923-a988-4d91-ae36-9bb927ff97f9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:57:30.865987 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed70923-a988-4d91-ae36-9bb927ff97f9 service swift] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:57:30.874714 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1160/2302] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jul 21 22:57:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 21 22:57:40.780383 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=3 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:57:40.871239 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=9 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:58:07.491447 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e0e12c-ede8-43f6-b2c0-3059f0084109 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:58:07.491447 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e0e12c-ede8-43f6-b2c0-3059f0084109 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:58:07.491447 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e0e12c-ede8-43f6-b2c0-3059f0084109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:58:07.511869 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-41e0e12c-ede8-43f6-b2c0-3059f0084109 None None] MFA Rules not processed for user `d10c68b7fd4446feb759832913fea3b3`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:58:07.513089 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-41e0e12c-ede8-43f6-b2c0-3059f0084109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:58:07.573507 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1143/2303] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:58:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:58:11.817287 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed70923-a988-4d91-ae36-9bb927ff97f9 service swift] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:58:11.873763 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed70923-a988-4d91-ae36-9bb927ff97f9 service swift] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:58:11.877437 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472c1e36-4a98-444c-b9a3-81a726805214 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:58:11.878114 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c1e36-4a98-444c-b9a3-81a726805214 service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:58:11.878397 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c1e36-4a98-444c-b9a3-81a726805214 service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:58:11.878672 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c1e36-4a98-444c-b9a3-81a726805214 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:58:11.880505 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-472c1e36-4a98-444c-b9a3-81a726805214 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:58:11.950377 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472c1e36-4a98-444c-b9a3-81a726805214 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:58:11.951861 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472c1e36-4a98-444c-b9a3-81a726805214 service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:58:11.963250 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1161/2304] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:58:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:58:12.270608 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e0e12c-ede8-43f6-b2c0-3059f0084109 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:58:12.272110 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-41e0e12c-ede8-43f6-b2c0-3059f0084109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:58:12.284116 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e0e12c-ede8-43f6-b2c0-3059f0084109 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:58:12.287757 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543af240-c696-4bbc-9641-ee07706f31c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:58:12.288399 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543af240-c696-4bbc-9641-ee07706f31c5 service neutron] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:58:12.288689 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543af240-c696-4bbc-9641-ee07706f31c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:58:12.288971 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543af240-c696-4bbc-9641-ee07706f31c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:58:12.291457 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-543af240-c696-4bbc-9641-ee07706f31c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:58:12.307119 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543af240-c696-4bbc-9641-ee07706f31c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:58:12.308763 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543af240-c696-4bbc-9641-ee07706f31c5 service neutron] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:58:12.322372 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1144/2305] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:58:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:58:12.763935 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472c1e36-4a98-444c-b9a3-81a726805214 service neutron] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:58:12.773179 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472c1e36-4a98-444c-b9a3-81a726805214 service neutron] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:58:12.776962 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ca7088-560c-49d8-be17-b67e56418592 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:58:12.777602 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ca7088-560c-49d8-be17-b67e56418592 service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:58:12.777900 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ca7088-560c-49d8-be17-b67e56418592 service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:58:12.778170 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ca7088-560c-49d8-be17-b67e56418592 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:58:12.788032 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ca7088-560c-49d8-be17-b67e56418592 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:58:12.789464 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ca7088-560c-49d8-be17-b67e56418592 service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:58:12.803012 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1162/2306] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:58:12 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:58:17.309926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c28dbc-d981-4657-ac19-b1d9ae4d3e23 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:58:17.310453 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c28dbc-d981-4657-ac19-b1d9ae4d3e23 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:58:17.310453 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c28dbc-d981-4657-ac19-b1d9ae4d3e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:58:17.330330 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-25c28dbc-d981-4657-ac19-b1d9ae4d3e23 None None] MFA Rules not processed for user `4947466838974a71815561640fa55c14`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:58:17.332056 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-25c28dbc-d981-4657-ac19-b1d9ae4d3e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:58:17.407370 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1145/2307] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:58:17 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:58:22.795364 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=60 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:58:23.924448 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ca7088-560c-49d8-be17-b67e56418592 service neutron] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:58:23.930337 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-93ca7088-560c-49d8-be17-b67e56418592 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:58:24.006634 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ca7088-560c-49d8-be17-b67e56418592 service neutron] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:58:24.011530 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084e08c5-264b-42a0-a88c-d3bd084726a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:58:24.012531 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e08c5-264b-42a0-a88c-d3bd084726a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:58:24.012895 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e08c5-264b-42a0-a88c-d3bd084726a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:58:24.013247 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e08c5-264b-42a0-a88c-d3bd084726a1 admin admin] PATH_INFO: `/v3/users/5eadb243fd6a4517bca72e18dec30e7a` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:58:24.015472 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084e08c5-264b-42a0-a88c-d3bd084726a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5eadb243fd6a4517bca72e18dec30e7a)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:58:24.017491 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084e08c5-264b-42a0-a88c-d3bd084726a1 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:58:24.206003 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-084e08c5-264b-42a0-a88c-d3bd084726a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5eadb243fd6a4517bca72e18dec30e7a'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:58:24.213165 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-084e08c5-264b-42a0-a88c-d3bd084726a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5eadb243fd6a4517bca72e18dec30e7a'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:58:24.213862 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-084e08c5-264b-42a0-a88c-d3bd084726a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5eadb243fd6a4517bca72e18dec30e7a'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:58:24.226867 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-084e08c5-264b-42a0-a88c-d3bd084726a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5eadb243fd6a4517bca72e18dec30e7a'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:58:24.317348 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-084e08c5-264b-42a0-a88c-d3bd084726a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5eadb243fd6a4517bca72e18dec30e7a'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:58:24.323935 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1163/2308] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:58:23 2022] DELETE /identity/v3/users/5eadb243fd6a4517bca72e18dec30e7a => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:58:24.599131 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c28dbc-d981-4657-ac19-b1d9ae4d3e23 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:58:24.610586 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c28dbc-d981-4657-ac19-b1d9ae4d3e23 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:58:24.668278 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd2f27-900c-42c7-a762-c935d5a0fb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:58:24.669142 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd2f27-900c-42c7-a762-c935d5a0fb81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:58:24.669429 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd2f27-900c-42c7-a762-c935d5a0fb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:58:24.669711 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd2f27-900c-42c7-a762-c935d5a0fb81 admin admin] PATH_INFO: `/v3/projects/2822a8cb96b94bf3bf525b0785c36934` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:58:24.670957 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fd2f27-900c-42c7-a762-c935d5a0fb81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2822a8cb96b94bf3bf525b0785c36934)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:58:24.672868 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fd2f27-900c-42c7-a762-c935d5a0fb81 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:58:24.787573 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:58:24.794231 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-a0fd2f27-900c-42c7-a762-c935d5a0fb81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2822a8cb96b94bf3bf525b0785c36934', 'request_id': 'req-a0fd2f27-900c-42c7-a762-c935d5a0fb81'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:58:24.800331 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-a0fd2f27-900c-42c7-a762-c935d5a0fb81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2822a8cb96b94bf3bf525b0785c36934', 'request_id': 'req-a0fd2f27-900c-42c7-a762-c935d5a0fb81'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:58:24.868126 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084e08c5-264b-42a0-a88c-d3bd084726a1 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:58:24.869600 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-084e08c5-264b-42a0-a88c-d3bd084726a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:58:24.871173 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-a0fd2f27-900c-42c7-a762-c935d5a0fb81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2822a8cb96b94bf3bf525b0785c36934', 'request_id': 'req-a0fd2f27-900c-42c7-a762-c935d5a0fb81'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:58:24.884213 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-a0fd2f27-900c-42c7-a762-c935d5a0fb81 admin admin] The token cache is being invalidate because project 2822a8cb96b94bf3bf525b0785c36934 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:58:24.884502 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-a0fd2f27-900c-42c7-a762-c935d5a0fb81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:58:24.887603 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-a0fd2f27-900c-42c7-a762-c935d5a0fb81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:58:24.887603 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1146/2309] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:58:24 2022] DELETE /identity/v3/projects/2822a8cb96b94bf3bf525b0785c36934 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:58:24.891688 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7335c9aa-c2eb-459e-be56-b9357e2a6db1 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:58:24.892035 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7335c9aa-c2eb-459e-be56-b9357e2a6db1 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:58:24.892327 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7335c9aa-c2eb-459e-be56-b9357e2a6db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:58:24.912267 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-7335c9aa-c2eb-459e-be56-b9357e2a6db1 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:58:24.913637 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-7335c9aa-c2eb-459e-be56-b9357e2a6db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:58:24.950266 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084e08c5-264b-42a0-a88c-d3bd084726a1 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:58:24.965871 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:58:24.987354 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1147/2310] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:58:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:58:24.995033 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfb7594-969c-4fb7-8214-435fa0955bbe None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:58:24.995688 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfb7594-969c-4fb7-8214-435fa0955bbe None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:58:24.996045 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfb7594-969c-4fb7-8214-435fa0955bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:58:25.015169 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-8dfb7594-969c-4fb7-8214-435fa0955bbe None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:58:25.016396 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-8dfb7594-969c-4fb7-8214-435fa0955bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:58:25.025026 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1148/2311] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:58:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:58:25.031712 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfb7594-969c-4fb7-8214-435fa0955bbe None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:58:25.038710 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd755670-04fa-4496-b926-13d3dbc78135 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bfdb655a2a44b33ab724daaf73a294b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:58:25.039478 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd755670-04fa-4496-b926-13d3dbc78135 service nova] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:58:25.039865 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd755670-04fa-4496-b926-13d3dbc78135 service nova] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:58:25.040253 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd755670-04fa-4496-b926-13d3dbc78135 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:58:25.042889 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-dd755670-04fa-4496-b926-13d3dbc78135 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:58:25.044162 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfb7594-969c-4fb7-8214-435fa0955bbe None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:58:25.050688 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc75ffcc-3780-4139-9da5-3a7ed83c8504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:58:25.051601 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc75ffcc-3780-4139-9da5-3a7ed83c8504 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:58:25.052163 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc75ffcc-3780-4139-9da5-3a7ed83c8504 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:58:25.052451 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc75ffcc-3780-4139-9da5-3a7ed83c8504 admin admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:58:25.053192 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc75ffcc-3780-4139-9da5-3a7ed83c8504 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:58:25.053502 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc75ffcc-3780-4139-9da5-3a7ed83c8504 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:58:25.055609 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc75ffcc-3780-4139-9da5-3a7ed83c8504 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:58:25.073881 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1149/2312] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 22:58:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 22:58:25.075001 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc75ffcc-3780-4139-9da5-3a7ed83c8504 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:58:25.088418 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc75ffcc-3780-4139-9da5-3a7ed83c8504 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:58:25.094757 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1782d36-e1fd-42b3-98b6-407d4dae5c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:58:25.096755 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1782d36-e1fd-42b3-98b6-407d4dae5c20 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:58:25.097052 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1782d36-e1fd-42b3-98b6-407d4dae5c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:58:25.097330 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1782d36-e1fd-42b3-98b6-407d4dae5c20 admin admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:58:25.098187 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1782d36-e1fd-42b3-98b6-407d4dae5c20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:58:25.100861 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1782d36-e1fd-42b3-98b6-407d4dae5c20 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:58:25.130497 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd755670-04fa-4496-b926-13d3dbc78135 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:58:25.132772 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd755670-04fa-4496-b926-13d3dbc78135 service nova] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:58:25.143475 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1164/2313] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:58:24 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:58:25.178511 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1150/2314] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:58:25 2022] POST /identity/v3/projects => generated 391 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:58:25.182735 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd755670-04fa-4496-b926-13d3dbc78135 service nova] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:58:25.194713 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd755670-04fa-4496-b926-13d3dbc78135 service nova] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:58:25.273508 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca63d4b-8b17-4ffd-b896-137b661c1084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:58:25.274333 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca63d4b-8b17-4ffd-b896-137b661c1084 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:58:25.274669 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca63d4b-8b17-4ffd-b896-137b661c1084 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:58:25.280787 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca63d4b-8b17-4ffd-b896-137b661c1084 admin admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:58:25.280787 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca63d4b-8b17-4ffd-b896-137b661c1084 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:58:25.280787 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca63d4b-8b17-4ffd-b896-137b661c1084 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:58:25.400527 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1165/2315] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:58:25 2022] POST /identity/v3/users => generated 296 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:58:25.404639 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1782d36-e1fd-42b3-98b6-407d4dae5c20 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:58:25.415034 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1782d36-e1fd-42b3-98b6-407d4dae5c20 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:58:25.419490 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5533b8-5b81-426c-a3ee-197c6a0fb2db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:58:25.420170 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5533b8-5b81-426c-a3ee-197c6a0fb2db admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:58:25.420435 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5533b8-5b81-426c-a3ee-197c6a0fb2db admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:58:25.420720 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5533b8-5b81-426c-a3ee-197c6a0fb2db admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:58:25.421443 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5533b8-5b81-426c-a3ee-197c6a0fb2db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:58:25.423069 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5533b8-5b81-426c-a3ee-197c6a0fb2db admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:58:25.432895 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1151/2316] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:58:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:58:25.436806 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca63d4b-8b17-4ffd-b896-137b661c1084 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:58:25.447280 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca63d4b-8b17-4ffd-b896-137b661c1084 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:58:25.452132 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1093d48a-c8c4-4b9b-ba6f-4a1744e4c667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:58:25.452788 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1093d48a-c8c4-4b9b-ba6f-4a1744e4c667 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:58:25.453076 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1093d48a-c8c4-4b9b-ba6f-4a1744e4c667 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:58:25.453357 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1093d48a-c8c4-4b9b-ba6f-4a1744e4c667 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:58:25.454090 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1093d48a-c8c4-4b9b-ba6f-4a1744e4c667 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:58:25.456342 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1093d48a-c8c4-4b9b-ba6f-4a1744e4c667 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:58:25.466412 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1166/2317] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:58:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:58:25.470255 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5533b8-5b81-426c-a3ee-197c6a0fb2db admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:58:25.480684 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5533b8-5b81-426c-a3ee-197c6a0fb2db admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:58:25.485206 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059e614c-f71c-43e3-ae39-3ee0cdfd3647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:58:25.486081 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e614c-f71c-43e3-ae39-3ee0cdfd3647 admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:58:25.486366 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e614c-f71c-43e3-ae39-3ee0cdfd3647 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:58:25.486641 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e614c-f71c-43e3-ae39-3ee0cdfd3647 admin admin] PATH_INFO: `/v3/projects/3b8f7f1282e345c6a4ae42c0ccf060dd/users/f98a3e599d5343d2a79804659d2dd1cb/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:58:25.501602 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059e614c-f71c-43e3-ae39-3ee0cdfd3647 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b8f7f1282e345c6a4ae42c0ccf060dd, user_id=f98a3e599d5343d2a79804659d2dd1cb, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:58:25.504831 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059e614c-f71c-43e3-ae39-3ee0cdfd3647 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:58:25.599239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1152/2318] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:58:25 2022] PUT /identity/v3/projects/3b8f7f1282e345c6a4ae42c0ccf060dd/users/f98a3e599d5343d2a79804659d2dd1cb/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:58:25.606897 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab4e13-9f56-4ab9-9edb-068e50c3d3ad None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:58:25.607303 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab4e13-9f56-4ab9-9edb-068e50c3d3ad None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:58:25.607651 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab4e13-9f56-4ab9-9edb-068e50c3d3ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:58:25.659286 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-51ab4e13-9f56-4ab9-9edb-068e50c3d3ad None None] MFA Rules not processed for user `f98a3e599d5343d2a79804659d2dd1cb`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:58:25.671391 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-51ab4e13-9f56-4ab9-9edb-068e50c3d3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:58:25.701769 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:58:25.753055 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1167/2319] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:58:25 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:58:25.764411 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059e614c-f71c-43e3-ae39-3ee0cdfd3647 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:58:25.766304 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-059e614c-f71c-43e3-ae39-3ee0cdfd3647 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:58:25.833483 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059e614c-f71c-43e3-ae39-3ee0cdfd3647 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:58:25.837487 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e56d0b-78e0-4c07-9ab2-9016d16670ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:58:25.838272 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e56d0b-78e0-4c07-9ab2-9016d16670ca service cinder] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:58:25.838624 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e56d0b-78e0-4c07-9ab2-9016d16670ca service cinder] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:58:25.839076 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e56d0b-78e0-4c07-9ab2-9016d16670ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:58:25.845216 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:58:25.849284 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e56d0b-78e0-4c07-9ab2-9016d16670ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:58:25.851151 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e56d0b-78e0-4c07-9ab2-9016d16670ca service cinder] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:58:25.862586 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1153/2320] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:58:25 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:58:28.539076 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ab4e13-9f56-4ab9-9edb-068e50c3d3ad None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:58:28.542164 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-51ab4e13-9f56-4ab9-9edb-068e50c3d3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:58:28.661639 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ab4e13-9f56-4ab9-9edb-068e50c3d3ad None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:58:28.667824 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d35dd9-86d6-49c8-ae95-2bcfd5ec75c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bfdb655a2a44b33ab724daaf73a294b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:58:28.668524 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d35dd9-86d6-49c8-ae95-2bcfd5ec75c7 service nova] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:58:28.668813 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d35dd9-86d6-49c8-ae95-2bcfd5ec75c7 service nova] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:58:28.669145 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d35dd9-86d6-49c8-ae95-2bcfd5ec75c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:58:28.671087 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-f8d35dd9-86d6-49c8-ae95-2bcfd5ec75c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:58:28.733934 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d35dd9-86d6-49c8-ae95-2bcfd5ec75c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:58:28.735582 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d35dd9-86d6-49c8-ae95-2bcfd5ec75c7 service nova] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:58:28.745509 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1168/2321] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:58:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:58:36.113938 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=3 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:58:36.539539 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7b412-cc30-4667-ad82-d06d403dfebc None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:58:36.539835 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7b412-cc30-4667-ad82-d06d403dfebc None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:58:36.540117 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7b412-cc30-4667-ad82-d06d403dfebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:58:36.559555 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-57e7b412-cc30-4667-ad82-d06d403dfebc None None] MFA Rules not processed for user `ef9ff560b7044ea2a18b3c3f8295937e`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:58:36.560747 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-57e7b412-cc30-4667-ad82-d06d403dfebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:58:36.567990 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1154/2322] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:58:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:58:37.419188 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d35dd9-86d6-49c8-ae95-2bcfd5ec75c7 service nova] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:58:37.420882 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-f8d35dd9-86d6-49c8-ae95-2bcfd5ec75c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:58:37.494620 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:58:37.500896 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d35dd9-86d6-49c8-ae95-2bcfd5ec75c7 service nova] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:58:37.506898 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6a3f0d-9ee7-4221-869c-3844db86406d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10c0bed169344a77bfad29624c5e5a8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:58:37.507707 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6a3f0d-9ee7-4221-869c-3844db86406d service placement] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:58:37.507994 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6a3f0d-9ee7-4221-869c-3844db86406d service placement] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:58:37.508267 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6a3f0d-9ee7-4221-869c-3844db86406d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:58:37.510095 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-1c6a3f0d-9ee7-4221-869c-3844db86406d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:58:37.524034 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6a3f0d-9ee7-4221-869c-3844db86406d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:58:37.525621 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6a3f0d-9ee7-4221-869c-3844db86406d service placement] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:58:37.536088 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1169/2323] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:58:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:58:40.461664 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e7b412-cc30-4667-ad82-d06d403dfebc None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:58:40.463221 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-57e7b412-cc30-4667-ad82-d06d403dfebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:58:40.530908 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e7b412-cc30-4667-ad82-d06d403dfebc None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:58:40.534738 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875cd8b8-ac98-4375-8677-1b897cd8b017 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:58:40.535483 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875cd8b8-ac98-4375-8677-1b897cd8b017 service neutron] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:58:40.535775 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875cd8b8-ac98-4375-8677-1b897cd8b017 service neutron] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:58:40.538977 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875cd8b8-ac98-4375-8677-1b897cd8b017 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:58:40.538977 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-875cd8b8-ac98-4375-8677-1b897cd8b017 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:58:40.548233 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875cd8b8-ac98-4375-8677-1b897cd8b017 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:58:40.549698 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875cd8b8-ac98-4375-8677-1b897cd8b017 service neutron] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:58:40.558473 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1155/2324] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:58:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:58:41.305139 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680c99f-a030-4bab-8985-7be2b35ef3fa None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:58:41.305763 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680c99f-a030-4bab-8985-7be2b35ef3fa None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:58:41.305763 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680c99f-a030-4bab-8985-7be2b35ef3fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:58:41.323758 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-3680c99f-a030-4bab-8985-7be2b35ef3fa None None] MFA Rules not processed for user `8c11dc2f4c764dc58458d9226efc878e`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:58:41.324978 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-3680c99f-a030-4bab-8985-7be2b35ef3fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:58:41.384100 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1170/2325] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:58:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:58:41.781939 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875cd8b8-ac98-4375-8677-1b897cd8b017 service neutron] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:58:41.783458 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-875cd8b8-ac98-4375-8677-1b897cd8b017 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:58:41.805943 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875cd8b8-ac98-4375-8677-1b897cd8b017 service neutron] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:58:41.819604 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c4e215-2c0d-499a-82c5-4c479dd37c28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:58:41.820513 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c4e215-2c0d-499a-82c5-4c479dd37c28 service neutron] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:58:41.820806 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c4e215-2c0d-499a-82c5-4c479dd37c28 service neutron] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:58:41.821090 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c4e215-2c0d-499a-82c5-4c479dd37c28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:58:41.823007 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-50c4e215-2c0d-499a-82c5-4c479dd37c28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:58:41.835194 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c4e215-2c0d-499a-82c5-4c479dd37c28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:58:41.836593 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c4e215-2c0d-499a-82c5-4c479dd37c28 service neutron] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:58:41.845313 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1156/2326] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:58:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:58:43.598249 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3680c99f-a030-4bab-8985-7be2b35ef3fa None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:58:43.616631 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3680c99f-a030-4bab-8985-7be2b35ef3fa None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:58:43.623257 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d326ec9-0384-4ba4-a05f-35e5da72e4dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bfdb655a2a44b33ab724daaf73a294b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:58:43.624033 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d326ec9-0384-4ba4-a05f-35e5da72e4dd service nova] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:58:43.624385 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d326ec9-0384-4ba4-a05f-35e5da72e4dd service nova] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:58:43.624753 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d326ec9-0384-4ba4-a05f-35e5da72e4dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:58:43.626632 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-3d326ec9-0384-4ba4-a05f-35e5da72e4dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:58:43.640586 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d326ec9-0384-4ba4-a05f-35e5da72e4dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:58:43.642306 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d326ec9-0384-4ba4-a05f-35e5da72e4dd service nova] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:58:43.654602 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1171/2327] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:58:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:58:47.193105 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c4e215-2c0d-499a-82c5-4c479dd37c28 service neutron] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:58:47.261769 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c4e215-2c0d-499a-82c5-4c479dd37c28 service neutron] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:58:47.266335 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e71c4e-a078-4a67-95da-f21e2d12b422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:58:47.267238 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e71c4e-a078-4a67-95da-f21e2d12b422 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:58:47.267550 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e71c4e-a078-4a67-95da-f21e2d12b422 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:58:47.268491 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e71c4e-a078-4a67-95da-f21e2d12b422 admin admin] PATH_INFO: `/v3/users/f98a3e599d5343d2a79804659d2dd1cb` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:58:47.269755 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e71c4e-a078-4a67-95da-f21e2d12b422 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f98a3e599d5343d2a79804659d2dd1cb)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:58:47.271829 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e71c4e-a078-4a67-95da-f21e2d12b422 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:58:47.532845 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-84e71c4e-a078-4a67-95da-f21e2d12b422 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f98a3e599d5343d2a79804659d2dd1cb'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:58:47.541145 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-84e71c4e-a078-4a67-95da-f21e2d12b422 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f98a3e599d5343d2a79804659d2dd1cb'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:58:47.543139 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-84e71c4e-a078-4a67-95da-f21e2d12b422 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f98a3e599d5343d2a79804659d2dd1cb'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:58:47.559484 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-84e71c4e-a078-4a67-95da-f21e2d12b422 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f98a3e599d5343d2a79804659d2dd1cb'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:58:47.560327 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-84e71c4e-a078-4a67-95da-f21e2d12b422 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f98a3e599d5343d2a79804659d2dd1cb'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:58:47.648931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1157/2328] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:58:47 2022] DELETE /identity/v3/users/f98a3e599d5343d2a79804659d2dd1cb => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:58:47.653610 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e105c66-4359-43b1-bf50-1e9f81fb1577 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:58:47.654052 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e105c66-4359-43b1-bf50-1e9f81fb1577 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:58:47.654288 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e105c66-4359-43b1-bf50-1e9f81fb1577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:58:47.680583 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-2e105c66-4359-43b1-bf50-1e9f81fb1577 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:58:47.681930 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-2e105c66-4359-43b1-bf50-1e9f81fb1577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:58:47.714952 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e71c4e-a078-4a67-95da-f21e2d12b422 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:58:47.716606 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-84e71c4e-a078-4a67-95da-f21e2d12b422 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:58:47.725169 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:58:47.758595 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1172/2329] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:58:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:58:47.770680 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e105c66-4359-43b1-bf50-1e9f81fb1577 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:58:47.772544 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-2e105c66-4359-43b1-bf50-1e9f81fb1577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:58:47.828776 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e71c4e-a078-4a67-95da-f21e2d12b422 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:58:47.835402 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee468c99-0460-48aa-a0af-6934ef7a9e01 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bfdb655a2a44b33ab724daaf73a294b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:58:47.836170 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee468c99-0460-48aa-a0af-6934ef7a9e01 service nova] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:58:47.836529 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee468c99-0460-48aa-a0af-6934ef7a9e01 service nova] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:58:47.836883 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee468c99-0460-48aa-a0af-6934ef7a9e01 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:58:47.839086 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-ee468c99-0460-48aa-a0af-6934ef7a9e01 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:58:47.846953 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e105c66-4359-43b1-bf50-1e9f81fb1577 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:58:47.851126 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c3de09-c2a1-4157-9d65-b94bc8e06ca5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:58:47.853038 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c3de09-c2a1-4157-9d65-b94bc8e06ca5 service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:58:47.853038 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c3de09-c2a1-4157-9d65-b94bc8e06ca5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:58:47.853038 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c3de09-c2a1-4157-9d65-b94bc8e06ca5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:58:47.857553 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee468c99-0460-48aa-a0af-6934ef7a9e01 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:58:47.859541 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee468c99-0460-48aa-a0af-6934ef7a9e01 service nova] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:58:47.863829 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c3de09-c2a1-4157-9d65-b94bc8e06ca5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:58:47.865565 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c3de09-c2a1-4157-9d65-b94bc8e06ca5 service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:58:47.873049 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1158/2330] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:58:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:58:47.885341 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1173/2331] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:58:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:58:48.386896 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee468c99-0460-48aa-a0af-6934ef7a9e01 service nova] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:58:48.389981 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-ee468c99-0460-48aa-a0af-6934ef7a9e01 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:58:48.403324 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee468c99-0460-48aa-a0af-6934ef7a9e01 service nova] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:58:48.408812 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e5d89-acf2-4701-b74b-b6aaf4f24138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:58:48.409831 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e5d89-acf2-4701-b74b-b6aaf4f24138 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:58:48.410125 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e5d89-acf2-4701-b74b-b6aaf4f24138 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:58:48.410402 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e5d89-acf2-4701-b74b-b6aaf4f24138 admin admin] PATH_INFO: `/v3/projects/3b8f7f1282e345c6a4ae42c0ccf060dd` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:58:48.411969 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054e5d89-acf2-4701-b74b-b6aaf4f24138 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b8f7f1282e345c6a4ae42c0ccf060dd)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:58:48.413885 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054e5d89-acf2-4701-b74b-b6aaf4f24138 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:58:48.559872 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-054e5d89-acf2-4701-b74b-b6aaf4f24138 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b8f7f1282e345c6a4ae42c0ccf060dd', 'request_id': 'req-054e5d89-acf2-4701-b74b-b6aaf4f24138'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:58:48.566814 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-054e5d89-acf2-4701-b74b-b6aaf4f24138 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b8f7f1282e345c6a4ae42c0ccf060dd', 'request_id': 'req-054e5d89-acf2-4701-b74b-b6aaf4f24138'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:58:48.674141 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-054e5d89-acf2-4701-b74b-b6aaf4f24138 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b8f7f1282e345c6a4ae42c0ccf060dd', 'request_id': 'req-054e5d89-acf2-4701-b74b-b6aaf4f24138'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:58:48.689934 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-054e5d89-acf2-4701-b74b-b6aaf4f24138 admin admin] The token cache is being invalidate because project 3b8f7f1282e345c6a4ae42c0ccf060dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:58:48.690267 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-054e5d89-acf2-4701-b74b-b6aaf4f24138 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:58:48.690951 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-054e5d89-acf2-4701-b74b-b6aaf4f24138 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:58:48.698821 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1159/2332] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:58:48 2022] DELETE /identity/v3/projects/3b8f7f1282e345c6a4ae42c0ccf060dd => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:58:55.992355 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c3de09-c2a1-4157-9d65-b94bc8e06ca5 service neutron] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:58:56.004358 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c3de09-c2a1-4157-9d65-b94bc8e06ca5 service neutron] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:58:56.027026 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:58:56.055520 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccf8a38-6313-4de4-b6cd-b6934d7ec8e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:58:56.056212 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccf8a38-6313-4de4-b6cd-b6934d7ec8e9 service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:58:56.056531 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccf8a38-6313-4de4-b6cd-b6934d7ec8e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:58:56.056811 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccf8a38-6313-4de4-b6cd-b6934d7ec8e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:58:56.058634 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-6ccf8a38-6313-4de4-b6cd-b6934d7ec8e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:58:56.068851 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccf8a38-6313-4de4-b6cd-b6934d7ec8e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:58:56.070298 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccf8a38-6313-4de4-b6cd-b6934d7ec8e9 service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:58:56.079371 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1174/2333] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:58:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:58:58.685656 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=60,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:59:06.075735 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=15 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:59:10.772685 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e5d89-acf2-4701-b74b-b6aaf4f24138 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:10.779593 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-054e5d89-acf2-4701-b74b-b6aaf4f24138 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:59:10.848692 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e5d89-acf2-4701-b74b-b6aaf4f24138 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:10.854651 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c9a59b-a792-4b36-951f-bdbbe47111dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bfdb655a2a44b33ab724daaf73a294b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:10.855974 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c9a59b-a792-4b36-951f-bdbbe47111dc service nova] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:10.856263 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c9a59b-a792-4b36-951f-bdbbe47111dc service nova] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:10.856545 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c9a59b-a792-4b36-951f-bdbbe47111dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:10.858849 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-66c9a59b-a792-4b36-951f-bdbbe47111dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:59:10.872122 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c9a59b-a792-4b36-951f-bdbbe47111dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:10.873749 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c9a59b-a792-4b36-951f-bdbbe47111dc service nova] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:10.885193 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1160/2334] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:59:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:59:11.191826 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccf8a38-6313-4de4-b6cd-b6934d7ec8e9 service neutron] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:11.193312 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-6ccf8a38-6313-4de4-b6cd-b6934d7ec8e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:59:11.251342 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccf8a38-6313-4de4-b6cd-b6934d7ec8e9 service neutron] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:11.255110 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bc77aa-7bf8-4861-ae88-c9d65d2b792b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:11.255786 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc77aa-7bf8-4861-ae88-c9d65d2b792b service cinder] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:11.256072 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc77aa-7bf8-4861-ae88-c9d65d2b792b service cinder] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:11.256346 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc77aa-7bf8-4861-ae88-c9d65d2b792b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:11.258701 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-35bc77aa-7bf8-4861-ae88-c9d65d2b792b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:59:11.318762 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bc77aa-7bf8-4861-ae88-c9d65d2b792b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:11.320224 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bc77aa-7bf8-4861-ae88-c9d65d2b792b service cinder] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:11.329333 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1175/2335] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:59:11 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:59:20.881178 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=34 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:59:21.325126 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=48 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:59:34.499860 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c9a59b-a792-4b36-951f-bdbbe47111dc service nova] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:34.501261 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-66c9a59b-a792-4b36-951f-bdbbe47111dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:59:34.512597 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c9a59b-a792-4b36-951f-bdbbe47111dc service nova] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:34.516226 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cc50d1-5932-4474-9c4d-50b6e4b4974d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:34.516854 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc50d1-5932-4474-9c4d-50b6e4b4974d service neutron] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:34.517138 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc50d1-5932-4474-9c4d-50b6e4b4974d service neutron] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:34.517416 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc50d1-5932-4474-9c4d-50b6e4b4974d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:34.519170 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-d6cc50d1-5932-4474-9c4d-50b6e4b4974d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:59:34.582432 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cc50d1-5932-4474-9c4d-50b6e4b4974d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:34.583880 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cc50d1-5932-4474-9c4d-50b6e4b4974d service neutron] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:34.593560 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1161/2336] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:59:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:59:38.117669 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bc77aa-7bf8-4861-ae88-c9d65d2b792b service cinder] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:38.119331 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-35bc77aa-7bf8-4861-ae88-c9d65d2b792b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:59:38.134752 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bc77aa-7bf8-4861-ae88-c9d65d2b792b service cinder] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:38.139706 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c77a6d5-6e8e-4ca9-b5f2-248a32237777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:38.140570 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c77a6d5-6e8e-4ca9-b5f2-248a32237777 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:38.140857 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c77a6d5-6e8e-4ca9-b5f2-248a32237777 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:38.141143 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c77a6d5-6e8e-4ca9-b5f2-248a32237777 admin admin] PATH_INFO: `/v3/users/4947466838974a71815561640fa55c14` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:38.142341 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c77a6d5-6e8e-4ca9-b5f2-248a32237777 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4947466838974a71815561640fa55c14)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:38.145016 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c77a6d5-6e8e-4ca9-b5f2-248a32237777 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:38.313556 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-8c77a6d5-6e8e-4ca9-b5f2-248a32237777 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4947466838974a71815561640fa55c14'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:59:38.321103 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-8c77a6d5-6e8e-4ca9-b5f2-248a32237777 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4947466838974a71815561640fa55c14'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:59:38.321779 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-8c77a6d5-6e8e-4ca9-b5f2-248a32237777 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4947466838974a71815561640fa55c14'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:59:38.334529 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-8c77a6d5-6e8e-4ca9-b5f2-248a32237777 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4947466838974a71815561640fa55c14'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:59:38.405923 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-8c77a6d5-6e8e-4ca9-b5f2-248a32237777 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4947466838974a71815561640fa55c14'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:59:38.412173 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1176/2337] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:59:38 2022] DELETE /identity/v3/users/4947466838974a71815561640fa55c14 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:59:38.653072 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cc50d1-5932-4474-9c4d-50b6e4b4974d service neutron] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:38.664850 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cc50d1-5932-4474-9c4d-50b6e4b4974d service neutron] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:38.724984 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598101fd-cf7d-422a-83e6-e4b476889e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:38.725787 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598101fd-cf7d-422a-83e6-e4b476889e74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:38.726335 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598101fd-cf7d-422a-83e6-e4b476889e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:38.726614 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598101fd-cf7d-422a-83e6-e4b476889e74 admin admin] PATH_INFO: `/v3/projects/755a70c4912f4795ae67cf9f036e02aa` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:38.727833 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598101fd-cf7d-422a-83e6-e4b476889e74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=755a70c4912f4795ae67cf9f036e02aa)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:38.729721 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598101fd-cf7d-422a-83e6-e4b476889e74 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:38.843944 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-598101fd-cf7d-422a-83e6-e4b476889e74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '755a70c4912f4795ae67cf9f036e02aa', 'request_id': 'req-598101fd-cf7d-422a-83e6-e4b476889e74'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:59:38.850050 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-598101fd-cf7d-422a-83e6-e4b476889e74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '755a70c4912f4795ae67cf9f036e02aa', 'request_id': 'req-598101fd-cf7d-422a-83e6-e4b476889e74'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:59:38.903172 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-598101fd-cf7d-422a-83e6-e4b476889e74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '755a70c4912f4795ae67cf9f036e02aa', 'request_id': 'req-598101fd-cf7d-422a-83e6-e4b476889e74'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:59:38.915578 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-598101fd-cf7d-422a-83e6-e4b476889e74 admin admin] The token cache is being invalidate because project 755a70c4912f4795ae67cf9f036e02aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:59:38.915806 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-598101fd-cf7d-422a-83e6-e4b476889e74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:59:38.916478 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-598101fd-cf7d-422a-83e6-e4b476889e74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:59:38.918004 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1162/2338] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:59:38 2022] DELETE /identity/v3/projects/755a70c4912f4795ae67cf9f036e02aa => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:59:38.929541 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee661c-1950-4861-91e2-4f12ac7e7b0d None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:38.929883 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee661c-1950-4861-91e2-4f12ac7e7b0d None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:38.930163 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee661c-1950-4861-91e2-4f12ac7e7b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:38.948232 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-9eee661c-1950-4861-91e2-4f12ac7e7b0d None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:59:38.949436 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-9eee661c-1950-4861-91e2-4f12ac7e7b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:59:39.009287 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1177/2339] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:59:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:59:39.017603 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6185db-7188-4c47-bf48-152bc3395dbf None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:39.017889 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6185db-7188-4c47-bf48-152bc3395dbf None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:39.018167 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6185db-7188-4c47-bf48-152bc3395dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:39.036467 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-0f6185db-7188-4c47-bf48-152bc3395dbf None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:59:39.037701 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-0f6185db-7188-4c47-bf48-152bc3395dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:59:39.045588 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1163/2340] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:59:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:59:39.049738 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee661c-1950-4861-91e2-4f12ac7e7b0d None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:39.060401 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee661c-1950-4861-91e2-4f12ac7e7b0d None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:39.065061 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff6e03-92c1-439b-afdd-1f529d790e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:39.065713 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff6e03-92c1-439b-afdd-1f529d790e97 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:39.065998 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff6e03-92c1-439b-afdd-1f529d790e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:39.066280 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff6e03-92c1-439b-afdd-1f529d790e97 admin admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:39.067066 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ff6e03-92c1-439b-afdd-1f529d790e97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:59:39.067374 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ff6e03-92c1-439b-afdd-1f529d790e97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:39.068967 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ff6e03-92c1-439b-afdd-1f529d790e97 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:39.080780 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1178/2341] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 22:59:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:59:39.085480 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6185db-7188-4c47-bf48-152bc3395dbf None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:39.094573 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6185db-7188-4c47-bf48-152bc3395dbf None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:39.099733 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3de6c09-4766-47fb-98f5-5a245939157a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:39.100373 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de6c09-4766-47fb-98f5-5a245939157a admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:39.100658 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de6c09-4766-47fb-98f5-5a245939157a admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:39.101192 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de6c09-4766-47fb-98f5-5a245939157a admin admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:39.101876 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3de6c09-4766-47fb-98f5-5a245939157a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:39.103800 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3de6c09-4766-47fb-98f5-5a245939157a admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:39.118875 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:59:39.219244 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1164/2342] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:59:39 2022] POST /identity/v3/projects => generated 371 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:59:39.223941 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff6e03-92c1-439b-afdd-1f529d790e97 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:39.234129 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff6e03-92c1-439b-afdd-1f529d790e97 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:39.289881 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641beb8a-f4d0-41f4-b628-66ba14c151cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:39.290560 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641beb8a-f4d0-41f4-b628-66ba14c151cc admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:39.290884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641beb8a-f4d0-41f4-b628-66ba14c151cc admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:39.291166 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641beb8a-f4d0-41f4-b628-66ba14c151cc admin admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:39.291858 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641beb8a-f4d0-41f4-b628-66ba14c151cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:39.293783 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641beb8a-f4d0-41f4-b628-66ba14c151cc admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:39.300488 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:59:39.388213 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1179/2343] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:59:39 2022] POST /identity/v3/users => generated 286 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:59:39.392373 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3de6c09-4766-47fb-98f5-5a245939157a admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:39.403111 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3de6c09-4766-47fb-98f5-5a245939157a admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:39.407960 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b721891f-6ff0-44a4-9002-15b34400d8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:39.408633 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b721891f-6ff0-44a4-9002-15b34400d8ec admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:39.408925 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b721891f-6ff0-44a4-9002-15b34400d8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:39.409205 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b721891f-6ff0-44a4-9002-15b34400d8ec admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:39.410555 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b721891f-6ff0-44a4-9002-15b34400d8ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:39.412234 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b721891f-6ff0-44a4-9002-15b34400d8ec admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:39.423401 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1165/2344] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:59:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:59:39.427420 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641beb8a-f4d0-41f4-b628-66ba14c151cc admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:39.437572 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641beb8a-f4d0-41f4-b628-66ba14c151cc admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:39.442934 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379fdd2b-ce9f-4812-87b2-c7667f922fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:39.443592 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379fdd2b-ce9f-4812-87b2-c7667f922fb4 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:39.443873 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379fdd2b-ce9f-4812-87b2-c7667f922fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:39.444145 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379fdd2b-ce9f-4812-87b2-c7667f922fb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:39.444869 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379fdd2b-ce9f-4812-87b2-c7667f922fb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:39.446519 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379fdd2b-ce9f-4812-87b2-c7667f922fb4 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:39.458971 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1180/2345] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:59:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:59:39.463093 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b721891f-6ff0-44a4-9002-15b34400d8ec admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:39.473193 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b721891f-6ff0-44a4-9002-15b34400d8ec admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:39.478639 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfe6d1f-7594-4d7f-9e27-4bdae582b459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:39.479162 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfe6d1f-7594-4d7f-9e27-4bdae582b459 admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:39.479506 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfe6d1f-7594-4d7f-9e27-4bdae582b459 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:39.479868 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfe6d1f-7594-4d7f-9e27-4bdae582b459 admin admin] PATH_INFO: `/v3/projects/6d78379423b94084a519adc7ae7be895/users/1c46be4fbec04311873d5baddc94ab6b/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:39.496046 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfe6d1f-7594-4d7f-9e27-4bdae582b459 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d78379423b94084a519adc7ae7be895, user_id=1c46be4fbec04311873d5baddc94ab6b, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:39.498829 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfe6d1f-7594-4d7f-9e27-4bdae582b459 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:39.572293 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1166/2346] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:59:39 2022] PUT /identity/v3/projects/6d78379423b94084a519adc7ae7be895/users/1c46be4fbec04311873d5baddc94ab6b/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:59:39.583034 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca07a2-2a33-4160-9e77-2d400434d268 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:39.583327 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca07a2-2a33-4160-9e77-2d400434d268 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:39.583608 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca07a2-2a33-4160-9e77-2d400434d268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:39.609403 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-4aca07a2-2a33-4160-9e77-2d400434d268 None None] MFA Rules not processed for user `1c46be4fbec04311873d5baddc94ab6b`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:59:39.610657 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-4aca07a2-2a33-4160-9e77-2d400434d268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:59:39.673223 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1181/2347] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:59:39 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:59:39.677683 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfe6d1f-7594-4d7f-9e27-4bdae582b459 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:39.739440 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfe6d1f-7594-4d7f-9e27-4bdae582b459 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:39.744024 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e51c90d-4444-4a86-bc0f-9873648a9606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:39.744713 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51c90d-4444-4a86-bc0f-9873648a9606 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:39.744995 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51c90d-4444-4a86-bc0f-9873648a9606 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:39.745294 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51c90d-4444-4a86-bc0f-9873648a9606 admin admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:39.745979 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e51c90d-4444-4a86-bc0f-9873648a9606 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:39.747915 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e51c90d-4444-4a86-bc0f-9873648a9606 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:39.870682 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1167/2348] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:59:39 2022] POST /identity/v3/projects => generated 373 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:59:39.874695 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aca07a2-2a33-4160-9e77-2d400434d268 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:39.886255 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aca07a2-2a33-4160-9e77-2d400434d268 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:39.942723 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21101238-2e86-4429-a26d-c984c800a720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:39.943421 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21101238-2e86-4429-a26d-c984c800a720 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:39.943704 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21101238-2e86-4429-a26d-c984c800a720 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:39.943974 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21101238-2e86-4429-a26d-c984c800a720 admin admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:39.944660 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21101238-2e86-4429-a26d-c984c800a720 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:39.946572 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21101238-2e86-4429-a26d-c984c800a720 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:40.045896 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1182/2349] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:59:39 2022] POST /identity/v3/users => generated 293 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:59:40.047150 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e51c90d-4444-4a86-bc0f-9873648a9606 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:40.060711 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e51c90d-4444-4a86-bc0f-9873648a9606 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:40.065545 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdafbf1-3240-4045-9c2b-501782257afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:40.066735 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdafbf1-3240-4045-9c2b-501782257afe admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:40.067041 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdafbf1-3240-4045-9c2b-501782257afe admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:40.067451 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdafbf1-3240-4045-9c2b-501782257afe admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:40.068222 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdafbf1-3240-4045-9c2b-501782257afe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:40.069852 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdafbf1-3240-4045-9c2b-501782257afe admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:40.080675 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1168/2350] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:59:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:59:40.084787 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21101238-2e86-4429-a26d-c984c800a720 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:40.094898 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21101238-2e86-4429-a26d-c984c800a720 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:40.099564 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295b8aaa-f57b-48a9-a92b-3659d21d24a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:40.100443 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295b8aaa-f57b-48a9-a92b-3659d21d24a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:40.100755 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295b8aaa-f57b-48a9-a92b-3659d21d24a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:40.101292 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295b8aaa-f57b-48a9-a92b-3659d21d24a2 admin admin] PATH_INFO: `/v3/projects/12bd8a9329dd49bd88c905650622521d/users/71737bcdcb0d4ef4846d27f1b1403bd9/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:40.116583 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295b8aaa-f57b-48a9-a92b-3659d21d24a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12bd8a9329dd49bd88c905650622521d, user_id=71737bcdcb0d4ef4846d27f1b1403bd9, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:40.119270 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295b8aaa-f57b-48a9-a92b-3659d21d24a2 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:40.197674 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1183/2351] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:59:40 2022] PUT /identity/v3/projects/12bd8a9329dd49bd88c905650622521d/users/71737bcdcb0d4ef4846d27f1b1403bd9/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:59:40.201605 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdafbf1-3240-4045-9c2b-501782257afe admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:40.211431 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdafbf1-3240-4045-9c2b-501782257afe admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:40.267531 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c432d7f-e267-42b3-9a8b-c3e4d50b60c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:40.268128 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c432d7f-e267-42b3-9a8b-c3e4d50b60c0 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:40.268417 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c432d7f-e267-42b3-9a8b-c3e4d50b60c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:40.268697 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c432d7f-e267-42b3-9a8b-c3e4d50b60c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:40.269437 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c432d7f-e267-42b3-9a8b-c3e4d50b60c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:40.271063 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c432d7f-e267-42b3-9a8b-c3e4d50b60c0 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:40.280822 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1169/2352] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:59:40 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:59:40.285432 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295b8aaa-f57b-48a9-a92b-3659d21d24a2 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:40.295674 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295b8aaa-f57b-48a9-a92b-3659d21d24a2 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:40.301099 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6890fe1e-66ca-41b0-b2d1-f93f9646d54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:40.301990 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890fe1e-66ca-41b0-b2d1-f93f9646d54d admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:40.302269 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890fe1e-66ca-41b0-b2d1-f93f9646d54d admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:40.302543 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890fe1e-66ca-41b0-b2d1-f93f9646d54d admin admin] PATH_INFO: `/v3/projects/12bd8a9329dd49bd88c905650622521d/users/71737bcdcb0d4ef4846d27f1b1403bd9/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:40.304548 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6890fe1e-66ca-41b0-b2d1-f93f9646d54d admin admin] RBAC: Authorizing `identity:create_grant(project_id=12bd8a9329dd49bd88c905650622521d, user_id=71737bcdcb0d4ef4846d27f1b1403bd9, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:40.307195 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6890fe1e-66ca-41b0-b2d1-f93f9646d54d admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:40.317224 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1184/2353] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:59:40 2022] PUT /identity/v3/projects/12bd8a9329dd49bd88c905650622521d/users/71737bcdcb0d4ef4846d27f1b1403bd9/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:59:40.327174 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922fd640-d5e0-4b21-a0c4-7fc7ea169a69 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:40.327471 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922fd640-d5e0-4b21-a0c4-7fc7ea169a69 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:40.327757 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922fd640-d5e0-4b21-a0c4-7fc7ea169a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:40.354136 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-922fd640-d5e0-4b21-a0c4-7fc7ea169a69 None None] MFA Rules not processed for user `71737bcdcb0d4ef4846d27f1b1403bd9`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:59:40.355379 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-922fd640-d5e0-4b21-a0c4-7fc7ea169a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:59:40.373812 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:59:40.422626 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1170/2354] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:59:40 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:59:40.434443 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6890fe1e-66ca-41b0-b2d1-f93f9646d54d admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:40.436331 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-6890fe1e-66ca-41b0-b2d1-f93f9646d54d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:59:40.458728 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:59:40.495405 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6890fe1e-66ca-41b0-b2d1-f93f9646d54d admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:40.499462 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97450ea0-4b21-404b-8ae5-2011551b1758 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:40.500114 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97450ea0-4b21-404b-8ae5-2011551b1758 service cinder] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:40.503739 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97450ea0-4b21-404b-8ae5-2011551b1758 service cinder] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:40.503739 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97450ea0-4b21-404b-8ae5-2011551b1758 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:40.510011 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97450ea0-4b21-404b-8ae5-2011551b1758 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:40.511474 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97450ea0-4b21-404b-8ae5-2011551b1758 service cinder] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:40.520970 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1185/2355] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:59:40 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:59:41.429108 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922fd640-d5e0-4b21-a0c4-7fc7ea169a69 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:41.491382 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922fd640-d5e0-4b21-a0c4-7fc7ea169a69 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:41.495922 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a406f0d-72f8-4896-bba5-5c4f4bf63215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:41.496794 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a406f0d-72f8-4896-bba5-5c4f4bf63215 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:41.497082 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a406f0d-72f8-4896-bba5-5c4f4bf63215 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:41.501540 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a406f0d-72f8-4896-bba5-5c4f4bf63215 admin admin] PATH_INFO: `/v3/users/1c46be4fbec04311873d5baddc94ab6b` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:41.501540 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a406f0d-72f8-4896-bba5-5c4f4bf63215 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c46be4fbec04311873d5baddc94ab6b)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:41.501540 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a406f0d-72f8-4896-bba5-5c4f4bf63215 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:41.684013 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-1a406f0d-72f8-4896-bba5-5c4f4bf63215 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c46be4fbec04311873d5baddc94ab6b'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:59:41.691809 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-1a406f0d-72f8-4896-bba5-5c4f4bf63215 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c46be4fbec04311873d5baddc94ab6b'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:59:41.692480 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-1a406f0d-72f8-4896-bba5-5c4f4bf63215 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c46be4fbec04311873d5baddc94ab6b'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:59:41.705010 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-1a406f0d-72f8-4896-bba5-5c4f4bf63215 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c46be4fbec04311873d5baddc94ab6b'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:59:41.705663 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-1a406f0d-72f8-4896-bba5-5c4f4bf63215 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c46be4fbec04311873d5baddc94ab6b'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:59:41.794291 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1171/2356] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:59:41 2022] DELETE /identity/v3/users/1c46be4fbec04311873d5baddc94ab6b => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:59:41.798884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0adb87-36ca-48f5-85e2-6dfd092785f4 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:41.799376 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0adb87-36ca-48f5-85e2-6dfd092785f4 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:41.799508 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0adb87-36ca-48f5-85e2-6dfd092785f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:41.817038 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-af0adb87-36ca-48f5-85e2-6dfd092785f4 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:59:41.818243 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-af0adb87-36ca-48f5-85e2-6dfd092785f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:59:41.881157 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1186/2357] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:59:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:59:41.890841 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a406f0d-72f8-4896-bba5-5c4f4bf63215 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:41.892345 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-1a406f0d-72f8-4896-bba5-5c4f4bf63215 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:59:41.932518 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:59:41.949655 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a406f0d-72f8-4896-bba5-5c4f4bf63215 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:41.953492 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b5a27c-7d78-4d52-9206-7e957fd72c42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:41.954153 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b5a27c-7d78-4d52-9206-7e957fd72c42 service neutron] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:41.954437 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b5a27c-7d78-4d52-9206-7e957fd72c42 service neutron] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:41.954715 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b5a27c-7d78-4d52-9206-7e957fd72c42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:41.965259 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b5a27c-7d78-4d52-9206-7e957fd72c42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:41.966690 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b5a27c-7d78-4d52-9206-7e957fd72c42 service neutron] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:41.976740 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1172/2358] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:59:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:59:42.658463 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0adb87-36ca-48f5-85e2-6dfd092785f4 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:42.660549 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-af0adb87-36ca-48f5-85e2-6dfd092785f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:59:42.673491 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0adb87-36ca-48f5-85e2-6dfd092785f4 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:42.679049 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0fa328-276b-48b6-8bff-4abb1de421d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:42.680037 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0fa328-276b-48b6-8bff-4abb1de421d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:42.680327 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0fa328-276b-48b6-8bff-4abb1de421d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:42.680609 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0fa328-276b-48b6-8bff-4abb1de421d2 admin admin] PATH_INFO: `/v3/projects/6d78379423b94084a519adc7ae7be895` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:42.681877 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0fa328-276b-48b6-8bff-4abb1de421d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d78379423b94084a519adc7ae7be895)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:42.683864 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0fa328-276b-48b6-8bff-4abb1de421d2 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:42.803187 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-4a0fa328-276b-48b6-8bff-4abb1de421d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d78379423b94084a519adc7ae7be895', 'request_id': 'req-4a0fa328-276b-48b6-8bff-4abb1de421d2'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:59:42.808711 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-4a0fa328-276b-48b6-8bff-4abb1de421d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d78379423b94084a519adc7ae7be895', 'request_id': 'req-4a0fa328-276b-48b6-8bff-4abb1de421d2'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:59:42.880011 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-4a0fa328-276b-48b6-8bff-4abb1de421d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d78379423b94084a519adc7ae7be895', 'request_id': 'req-4a0fa328-276b-48b6-8bff-4abb1de421d2'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:59:42.891191 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-4a0fa328-276b-48b6-8bff-4abb1de421d2 admin admin] The token cache is being invalidate because project 6d78379423b94084a519adc7ae7be895 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:59:42.891513 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-4a0fa328-276b-48b6-8bff-4abb1de421d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:59:42.892111 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-4a0fa328-276b-48b6-8bff-4abb1de421d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:59:42.893648 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1187/2359] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:59:42 2022] DELETE /identity/v3/projects/6d78379423b94084a519adc7ae7be895 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:59:42.898092 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b5a27c-7d78-4d52-9206-7e957fd72c42 service neutron] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:42.907775 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b5a27c-7d78-4d52-9206-7e957fd72c42 service neutron] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:42.968630 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ce762b-6cfa-4b76-86d7-b1be2525c9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:42.969489 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ce762b-6cfa-4b76-86d7-b1be2525c9b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:42.969768 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ce762b-6cfa-4b76-86d7-b1be2525c9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:42.970039 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ce762b-6cfa-4b76-86d7-b1be2525c9b5 admin admin] PATH_INFO: `/v3/users/71737bcdcb0d4ef4846d27f1b1403bd9` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:42.971228 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ce762b-6cfa-4b76-86d7-b1be2525c9b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71737bcdcb0d4ef4846d27f1b1403bd9)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:42.973772 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ce762b-6cfa-4b76-86d7-b1be2525c9b5 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:43.115980 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-f6ce762b-6cfa-4b76-86d7-b1be2525c9b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71737bcdcb0d4ef4846d27f1b1403bd9'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:59:43.122958 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-f6ce762b-6cfa-4b76-86d7-b1be2525c9b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71737bcdcb0d4ef4846d27f1b1403bd9'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:59:43.123617 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-f6ce762b-6cfa-4b76-86d7-b1be2525c9b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71737bcdcb0d4ef4846d27f1b1403bd9'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:59:43.135553 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-f6ce762b-6cfa-4b76-86d7-b1be2525c9b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71737bcdcb0d4ef4846d27f1b1403bd9'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:59:43.136184 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-f6ce762b-6cfa-4b76-86d7-b1be2525c9b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71737bcdcb0d4ef4846d27f1b1403bd9'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:59:43.211685 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1173/2360] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:59:42 2022] DELETE /identity/v3/users/71737bcdcb0d4ef4846d27f1b1403bd9 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:59:43.622202 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0fa328-276b-48b6-8bff-4abb1de421d2 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:43.623689 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-4a0fa328-276b-48b6-8bff-4abb1de421d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:59:43.637322 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0fa328-276b-48b6-8bff-4abb1de421d2 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:43.683049 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:59:43.696375 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9086d832-5380-4399-ab8f-2fa3d6fcf4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:43.697235 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9086d832-5380-4399-ab8f-2fa3d6fcf4d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:43.697518 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9086d832-5380-4399-ab8f-2fa3d6fcf4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:43.697791 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9086d832-5380-4399-ab8f-2fa3d6fcf4d3 admin admin] PATH_INFO: `/v3/projects/12bd8a9329dd49bd88c905650622521d` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:43.698996 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9086d832-5380-4399-ab8f-2fa3d6fcf4d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12bd8a9329dd49bd88c905650622521d)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:43.703661 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9086d832-5380-4399-ab8f-2fa3d6fcf4d3 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:43.834471 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-9086d832-5380-4399-ab8f-2fa3d6fcf4d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12bd8a9329dd49bd88c905650622521d', 'request_id': 'req-9086d832-5380-4399-ab8f-2fa3d6fcf4d3'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:59:43.839813 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-9086d832-5380-4399-ab8f-2fa3d6fcf4d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12bd8a9329dd49bd88c905650622521d', 'request_id': 'req-9086d832-5380-4399-ab8f-2fa3d6fcf4d3'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:59:43.911152 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-9086d832-5380-4399-ab8f-2fa3d6fcf4d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12bd8a9329dd49bd88c905650622521d', 'request_id': 'req-9086d832-5380-4399-ab8f-2fa3d6fcf4d3'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:59:43.922329 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-9086d832-5380-4399-ab8f-2fa3d6fcf4d3 admin admin] The token cache is being invalidate because project 12bd8a9329dd49bd88c905650622521d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:59:43.922701 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-9086d832-5380-4399-ab8f-2fa3d6fcf4d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:59:43.923337 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-9086d832-5380-4399-ab8f-2fa3d6fcf4d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:59:43.924889 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1188/2361] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:59:43 2022] DELETE /identity/v3/projects/12bd8a9329dd49bd88c905650622521d => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:59:43.930343 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d268386-8a97-4650-912d-7fb8b4cd57f6 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:43.930839 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d268386-8a97-4650-912d-7fb8b4cd57f6 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:43.930936 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d268386-8a97-4650-912d-7fb8b4cd57f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:43.949629 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-8d268386-8a97-4650-912d-7fb8b4cd57f6 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:59:43.950756 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-8d268386-8a97-4650-912d-7fb8b4cd57f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:59:44.002074 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:59:44.011436 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1174/2362] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:59:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:59:44.020389 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad8df2-07df-4b4b-8eaf-16b39be6cb82 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:44.020678 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad8df2-07df-4b4b-8eaf-16b39be6cb82 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:44.020958 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad8df2-07df-4b4b-8eaf-16b39be6cb82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:44.039158 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-78ad8df2-07df-4b4b-8eaf-16b39be6cb82 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:59:44.040356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-78ad8df2-07df-4b4b-8eaf-16b39be6cb82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:59:44.048433 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1189/2363] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:59:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:59:44.057833 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d268386-8a97-4650-912d-7fb8b4cd57f6 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:44.068484 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d268386-8a97-4650-912d-7fb8b4cd57f6 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:44.074469 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ade72f-9d36-4090-a44a-b52dd48eb656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:44.075169 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ade72f-9d36-4090-a44a-b52dd48eb656 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:44.075453 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ade72f-9d36-4090-a44a-b52dd48eb656 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:44.075725 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ade72f-9d36-4090-a44a-b52dd48eb656 admin admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:44.076473 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ade72f-9d36-4090-a44a-b52dd48eb656 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:59:44.076776 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ade72f-9d36-4090-a44a-b52dd48eb656 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:44.078354 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ade72f-9d36-4090-a44a-b52dd48eb656 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:44.092404 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1175/2364] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 22:59:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:59:44.096674 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ad8df2-07df-4b4b-8eaf-16b39be6cb82 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:44.106792 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ad8df2-07df-4b4b-8eaf-16b39be6cb82 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:44.111449 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9e19bb-4170-4f41-b117-81418f1e5518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:44.112092 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e19bb-4170-4f41-b117-81418f1e5518 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:44.112375 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e19bb-4170-4f41-b117-81418f1e5518 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:44.112648 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e19bb-4170-4f41-b117-81418f1e5518 admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:44.113966 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9e19bb-4170-4f41-b117-81418f1e5518 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:44.115875 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9e19bb-4170-4f41-b117-81418f1e5518 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:44.236096 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1190/2365] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:59:44 2022] POST /identity/v3/projects => generated 369 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:59:44.240939 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ade72f-9d36-4090-a44a-b52dd48eb656 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:44.251178 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ade72f-9d36-4090-a44a-b52dd48eb656 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:44.310032 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4eb7fe-64c2-4239-b312-6b98d1b8fb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:44.310701 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4eb7fe-64c2-4239-b312-6b98d1b8fb0a admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:44.311012 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4eb7fe-64c2-4239-b312-6b98d1b8fb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:44.311289 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4eb7fe-64c2-4239-b312-6b98d1b8fb0a admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:44.311978 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4eb7fe-64c2-4239-b312-6b98d1b8fb0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:44.313921 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4eb7fe-64c2-4239-b312-6b98d1b8fb0a admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:44.400638 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1176/2366] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:59:44 2022] POST /identity/v3/users => generated 285 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:59:44.404706 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9e19bb-4170-4f41-b117-81418f1e5518 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:44.416064 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9e19bb-4170-4f41-b117-81418f1e5518 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:44.420865 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321a7dc3-5ca4-4116-9e61-a33aac66a283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:44.421573 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a7dc3-5ca4-4116-9e61-a33aac66a283 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:44.421858 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a7dc3-5ca4-4116-9e61-a33aac66a283 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:44.422135 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a7dc3-5ca4-4116-9e61-a33aac66a283 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:44.422894 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321a7dc3-5ca4-4116-9e61-a33aac66a283 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:44.424491 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321a7dc3-5ca4-4116-9e61-a33aac66a283 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:44.439190 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1191/2367] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:59:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:59:44.443725 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4eb7fe-64c2-4239-b312-6b98d1b8fb0a admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:44.453864 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4eb7fe-64c2-4239-b312-6b98d1b8fb0a admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:44.459034 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefde6de-b1ea-49c7-a9ef-202110d0bc51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:44.459690 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefde6de-b1ea-49c7-a9ef-202110d0bc51 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:44.459978 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefde6de-b1ea-49c7-a9ef-202110d0bc51 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:44.460255 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefde6de-b1ea-49c7-a9ef-202110d0bc51 admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:44.460994 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefde6de-b1ea-49c7-a9ef-202110d0bc51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:44.462554 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefde6de-b1ea-49c7-a9ef-202110d0bc51 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:44.473075 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1177/2368] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:59:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:59:44.477324 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321a7dc3-5ca4-4116-9e61-a33aac66a283 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:44.487414 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321a7dc3-5ca4-4116-9e61-a33aac66a283 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:44.491980 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7444be17-bf75-4784-9630-855f429c6794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:44.492886 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444be17-bf75-4784-9630-855f429c6794 admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:44.493166 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444be17-bf75-4784-9630-855f429c6794 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:44.493438 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444be17-bf75-4784-9630-855f429c6794 admin admin] PATH_INFO: `/v3/projects/f643a79596dd47c9a512865eee7a2b13/users/e8b054a2ce1848d79b93b12c99f896b5/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:44.508187 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7444be17-bf75-4784-9630-855f429c6794 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f643a79596dd47c9a512865eee7a2b13, user_id=e8b054a2ce1848d79b93b12c99f896b5, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:44.510839 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7444be17-bf75-4784-9630-855f429c6794 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:44.589193 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1192/2369] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:59:44 2022] PUT /identity/v3/projects/f643a79596dd47c9a512865eee7a2b13/users/e8b054a2ce1848d79b93b12c99f896b5/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:59:44.600102 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce5e579-6a54-4879-955f-1ef17dd27144 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:44.600445 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce5e579-6a54-4879-955f-1ef17dd27144 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:44.600731 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce5e579-6a54-4879-955f-1ef17dd27144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:44.628001 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-3ce5e579-6a54-4879-955f-1ef17dd27144 None None] MFA Rules not processed for user `e8b054a2ce1848d79b93b12c99f896b5`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:59:44.629520 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-3ce5e579-6a54-4879-955f-1ef17dd27144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:59:44.698583 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1178/2370] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:59:44 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 22:59:44.703133 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7444be17-bf75-4784-9630-855f429c6794 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:44.766572 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7444be17-bf75-4784-9630-855f429c6794 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:44.771884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e8bc94-f3d0-49bd-a582-e30e12284271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:44.772549 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e8bc94-f3d0-49bd-a582-e30e12284271 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:44.772837 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e8bc94-f3d0-49bd-a582-e30e12284271 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:44.773120 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e8bc94-f3d0-49bd-a582-e30e12284271 admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:44.773894 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e8bc94-f3d0-49bd-a582-e30e12284271 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:44.775823 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e8bc94-f3d0-49bd-a582-e30e12284271 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:44.868882 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:59:44.876865 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1193/2371] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:59:44 2022] POST /identity/v3/projects => generated 365 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:59:44.881756 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce5e579-6a54-4879-955f-1ef17dd27144 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:44.893200 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce5e579-6a54-4879-955f-1ef17dd27144 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:44.943417 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:59:44.953083 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dba972-cec3-49de-b012-6427601cab75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:44.953812 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dba972-cec3-49de-b012-6427601cab75 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:44.954099 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dba972-cec3-49de-b012-6427601cab75 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:44.954380 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dba972-cec3-49de-b012-6427601cab75 admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:44.955239 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dba972-cec3-49de-b012-6427601cab75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:44.957990 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dba972-cec3-49de-b012-6427601cab75 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:45.375005 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1179/2372] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:59:44 2022] POST /identity/v3/users => generated 289 bytes in 494 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:59:45.379157 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e8bc94-f3d0-49bd-a582-e30e12284271 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:45.389628 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e8bc94-f3d0-49bd-a582-e30e12284271 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:45.394800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42e0148-2904-4e50-9e3a-84ca8f20294b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:45.395512 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e0148-2904-4e50-9e3a-84ca8f20294b admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:45.395809 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e0148-2904-4e50-9e3a-84ca8f20294b admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:45.396081 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e0148-2904-4e50-9e3a-84ca8f20294b admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:45.396830 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42e0148-2904-4e50-9e3a-84ca8f20294b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:45.398546 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42e0148-2904-4e50-9e3a-84ca8f20294b admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:45.409541 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1194/2373] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:59:45.414100 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dba972-cec3-49de-b012-6427601cab75 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:45.424401 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dba972-cec3-49de-b012-6427601cab75 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:45.429147 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbd9e8c-cac4-4aed-9f68-62ec5090210c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:45.431576 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd9e8c-cac4-4aed-9f68-62ec5090210c admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:45.431576 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd9e8c-cac4-4aed-9f68-62ec5090210c admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:45.431576 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd9e8c-cac4-4aed-9f68-62ec5090210c admin admin] PATH_INFO: `/v3/projects/223f259dd9ad4f41b3212dad93b8bf29/users/d13f20e8d22b4f798cef08b52ff1c3ec/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:45.445764 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbd9e8c-cac4-4aed-9f68-62ec5090210c admin admin] RBAC: Authorizing `identity:create_grant(project_id=223f259dd9ad4f41b3212dad93b8bf29, user_id=d13f20e8d22b4f798cef08b52ff1c3ec, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:45.448518 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbd9e8c-cac4-4aed-9f68-62ec5090210c admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:45.500640 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1180/2374] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:59:45 2022] PUT /identity/v3/projects/223f259dd9ad4f41b3212dad93b8bf29/users/d13f20e8d22b4f798cef08b52ff1c3ec/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:59:45.505203 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42e0148-2904-4e50-9e3a-84ca8f20294b admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:45.515686 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42e0148-2904-4e50-9e3a-84ca8f20294b admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:45.572670 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0451da-2479-4064-9ae5-11382ac7facd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:45.573342 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0451da-2479-4064-9ae5-11382ac7facd admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:45.573632 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0451da-2479-4064-9ae5-11382ac7facd admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:45.573914 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0451da-2479-4064-9ae5-11382ac7facd admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:45.574641 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0451da-2479-4064-9ae5-11382ac7facd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:45.576268 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0451da-2479-4064-9ae5-11382ac7facd admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:45.589991 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1195/2375] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:59:45 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:59:45.595179 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbd9e8c-cac4-4aed-9f68-62ec5090210c admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:45.606987 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbd9e8c-cac4-4aed-9f68-62ec5090210c admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:45.611675 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390e50ee-7148-4948-bc65-5bb3a420c50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:45.612550 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e50ee-7148-4948-bc65-5bb3a420c50e admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:45.612857 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e50ee-7148-4948-bc65-5bb3a420c50e admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:45.613195 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e50ee-7148-4948-bc65-5bb3a420c50e admin admin] PATH_INFO: `/v3/projects/223f259dd9ad4f41b3212dad93b8bf29/users/d13f20e8d22b4f798cef08b52ff1c3ec/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:45.615251 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390e50ee-7148-4948-bc65-5bb3a420c50e admin admin] RBAC: Authorizing `identity:create_grant(project_id=223f259dd9ad4f41b3212dad93b8bf29, user_id=d13f20e8d22b4f798cef08b52ff1c3ec, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:45.617863 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390e50ee-7148-4948-bc65-5bb3a420c50e admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:45.629011 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1181/2376] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:59:45 2022] PUT /identity/v3/projects/223f259dd9ad4f41b3212dad93b8bf29/users/d13f20e8d22b4f798cef08b52ff1c3ec/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:59:45.640235 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358c3a1-78bf-4d87-9b3a-9e2dfb6c51a5 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:45.640547 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358c3a1-78bf-4d87-9b3a-9e2dfb6c51a5 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:45.640826 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358c3a1-78bf-4d87-9b3a-9e2dfb6c51a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:45.667491 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-4358c3a1-78bf-4d87-9b3a-9e2dfb6c51a5 None None] MFA Rules not processed for user `d13f20e8d22b4f798cef08b52ff1c3ec`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:59:45.668693 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-4358c3a1-78bf-4d87-9b3a-9e2dfb6c51a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:59:45.734900 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1196/2377] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:59:45 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:59:45.746013 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390e50ee-7148-4948-bc65-5bb3a420c50e admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:45.747715 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-390e50ee-7148-4948-bc65-5bb3a420c50e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:59:45.805545 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390e50ee-7148-4948-bc65-5bb3a420c50e admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:45.809183 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c54d3b-0a37-47d2-9278-d43caacba6b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:45.809826 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c54d3b-0a37-47d2-9278-d43caacba6b7 service cinder] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:45.810112 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c54d3b-0a37-47d2-9278-d43caacba6b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:45.810391 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c54d3b-0a37-47d2-9278-d43caacba6b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:45.820283 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c54d3b-0a37-47d2-9278-d43caacba6b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:45.821713 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c54d3b-0a37-47d2-9278-d43caacba6b7 service cinder] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:45.832257 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1182/2378] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:59:45 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:59:45.961371 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4358c3a1-78bf-4d87-9b3a-9e2dfb6c51a5 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:45.962861 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-4358c3a1-78bf-4d87-9b3a-9e2dfb6c51a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:59:45.974620 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4358c3a1-78bf-4d87-9b3a-9e2dfb6c51a5 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:45.978417 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b495cb-ad3d-4fbd-9169-d16202c1513b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:45.979097 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b495cb-ad3d-4fbd-9169-d16202c1513b service cinder] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:45.979375 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b495cb-ad3d-4fbd-9169-d16202c1513b service cinder] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:45.979651 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b495cb-ad3d-4fbd-9169-d16202c1513b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:46.036314 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b495cb-ad3d-4fbd-9169-d16202c1513b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:46.037755 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b495cb-ad3d-4fbd-9169-d16202c1513b service cinder] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:46.047088 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1197/2379] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:59:45 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:59:50.047387 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f05dd-6eaf-4d5b-8494-8b6c67778c7b None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:50.047870 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f05dd-6eaf-4d5b-8494-8b6c67778c7b None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:50.047977 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f05dd-6eaf-4d5b-8494-8b6c67778c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:50.066387 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-d76f05dd-6eaf-4d5b-8494-8b6c67778c7b None None] MFA Rules not processed for user `63cc904578554aea84ff1917a5ed075e`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:59:50.067665 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-d76f05dd-6eaf-4d5b-8494-8b6c67778c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:59:50.126661 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1183/2380] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jul 21 22:59:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:59:50.169223 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b495cb-ad3d-4fbd-9169-d16202c1513b service cinder] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:50.170687 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-21b495cb-ad3d-4fbd-9169-d16202c1513b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:59:50.180823 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:59:50.242279 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b495cb-ad3d-4fbd-9169-d16202c1513b service cinder] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:50.247287 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255788eb-347b-4bc4-a0e7-23f17357a444 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba7eef4e1b27476ab4016449d9c3a4b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:50.247927 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255788eb-347b-4bc4-a0e7-23f17357a444 service swift] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:50.248211 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255788eb-347b-4bc4-a0e7-23f17357a444 service swift] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:50.248482 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255788eb-347b-4bc4-a0e7-23f17357a444 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:50.257202 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255788eb-347b-4bc4-a0e7-23f17357a444 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:50.259099 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255788eb-347b-4bc4-a0e7-23f17357a444 service swift] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:50.267470 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1198/2381] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jul 21 22:59:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 22:59:51.273638 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f05dd-6eaf-4d5b-8494-8b6c67778c7b None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:51.320185 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:59:51.345172 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f05dd-6eaf-4d5b-8494-8b6c67778c7b None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:51.349843 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebe156d-ebf9-499d-be1c-e9f9b6f27244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:51.350739 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe156d-ebf9-499d-be1c-e9f9b6f27244 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:51.351052 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe156d-ebf9-499d-be1c-e9f9b6f27244 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:51.351592 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe156d-ebf9-499d-be1c-e9f9b6f27244 admin admin] PATH_INFO: `/v3/users/e8b054a2ce1848d79b93b12c99f896b5` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:51.364745 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebe156d-ebf9-499d-be1c-e9f9b6f27244 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8b054a2ce1848d79b93b12c99f896b5)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:51.364745 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebe156d-ebf9-499d-be1c-e9f9b6f27244 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:51.773582 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-8ebe156d-ebf9-499d-be1c-e9f9b6f27244 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8b054a2ce1848d79b93b12c99f896b5'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:59:51.785640 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-8ebe156d-ebf9-499d-be1c-e9f9b6f27244 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8b054a2ce1848d79b93b12c99f896b5'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:59:51.786379 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-8ebe156d-ebf9-499d-be1c-e9f9b6f27244 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8b054a2ce1848d79b93b12c99f896b5'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:59:51.807677 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-8ebe156d-ebf9-499d-be1c-e9f9b6f27244 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8b054a2ce1848d79b93b12c99f896b5'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:59:51.808523 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-8ebe156d-ebf9-499d-be1c-e9f9b6f27244 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8b054a2ce1848d79b93b12c99f896b5'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:59:51.892867 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1184/2382] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:59:51 2022] DELETE /identity/v3/users/e8b054a2ce1848d79b93b12c99f896b5 => generated 0 bytes in 649 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:59:51.895389 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b21a84-de91-407d-a730-08b28aa20ac1 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:51.902765 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b21a84-de91-407d-a730-08b28aa20ac1 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:51.902765 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b21a84-de91-407d-a730-08b28aa20ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:51.916571 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-67b21a84-de91-407d-a730-08b28aa20ac1 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:59:51.918202 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-67b21a84-de91-407d-a730-08b28aa20ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:59:52.015923 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1199/2383] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:59:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:59:52.025819 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebe156d-ebf9-499d-be1c-e9f9b6f27244 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:52.027359 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-8ebe156d-ebf9-499d-be1c-e9f9b6f27244 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:59:52.095642 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebe156d-ebf9-499d-be1c-e9f9b6f27244 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:52.099480 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cf3293-bab1-4c58-b347-5562dfe9aca7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:52.102815 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf3293-bab1-4c58-b347-5562dfe9aca7 service neutron] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:52.102815 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf3293-bab1-4c58-b347-5562dfe9aca7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:52.102815 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf3293-bab1-4c58-b347-5562dfe9aca7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:52.113315 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cf3293-bab1-4c58-b347-5562dfe9aca7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:52.114746 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cf3293-bab1-4c58-b347-5562dfe9aca7 service neutron] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:52.125146 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1185/2384] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:59:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:59:52.617919 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b21a84-de91-407d-a730-08b28aa20ac1 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:52.619701 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-67b21a84-de91-407d-a730-08b28aa20ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:59:52.633866 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b21a84-de91-407d-a730-08b28aa20ac1 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:52.639939 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b439b8-87ec-40b4-bb4e-8b14a37ff525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:52.643677 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b439b8-87ec-40b4-bb4e-8b14a37ff525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:52.643930 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b439b8-87ec-40b4-bb4e-8b14a37ff525 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:52.644374 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b439b8-87ec-40b4-bb4e-8b14a37ff525 admin admin] PATH_INFO: `/v3/projects/f643a79596dd47c9a512865eee7a2b13` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:52.645725 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b439b8-87ec-40b4-bb4e-8b14a37ff525 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f643a79596dd47c9a512865eee7a2b13)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:52.647905 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b439b8-87ec-40b4-bb4e-8b14a37ff525 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:52.781377 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-59b439b8-87ec-40b4-bb4e-8b14a37ff525 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f643a79596dd47c9a512865eee7a2b13', 'request_id': 'req-59b439b8-87ec-40b4-bb4e-8b14a37ff525'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:59:52.788081 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-59b439b8-87ec-40b4-bb4e-8b14a37ff525 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f643a79596dd47c9a512865eee7a2b13', 'request_id': 'req-59b439b8-87ec-40b4-bb4e-8b14a37ff525'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:59:52.852237 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-59b439b8-87ec-40b4-bb4e-8b14a37ff525 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f643a79596dd47c9a512865eee7a2b13', 'request_id': 'req-59b439b8-87ec-40b4-bb4e-8b14a37ff525'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:59:52.866835 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-59b439b8-87ec-40b4-bb4e-8b14a37ff525 admin admin] The token cache is being invalidate because project f643a79596dd47c9a512865eee7a2b13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:59:52.866835 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-59b439b8-87ec-40b4-bb4e-8b14a37ff525 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:59:52.866835 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-59b439b8-87ec-40b4-bb4e-8b14a37ff525 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:59:52.868280 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1200/2385] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:59:52 2022] DELETE /identity/v3/projects/f643a79596dd47c9a512865eee7a2b13 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:59:52.872524 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cf3293-bab1-4c58-b347-5562dfe9aca7 service neutron] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:52.888837 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cf3293-bab1-4c58-b347-5562dfe9aca7 service neutron] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:53.016769 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a412b72d-5c24-4662-9417-0169d9db780e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:53.016769 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412b72d-5c24-4662-9417-0169d9db780e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:53.016769 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412b72d-5c24-4662-9417-0169d9db780e admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:53.016769 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412b72d-5c24-4662-9417-0169d9db780e admin admin] PATH_INFO: `/v3/users/d13f20e8d22b4f798cef08b52ff1c3ec` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:53.017451 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a412b72d-5c24-4662-9417-0169d9db780e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d13f20e8d22b4f798cef08b52ff1c3ec)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:53.019245 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a412b72d-5c24-4662-9417-0169d9db780e admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:53.041703 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:59:53.414046 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-a412b72d-5c24-4662-9417-0169d9db780e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd13f20e8d22b4f798cef08b52ff1c3ec'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:59:53.422002 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-a412b72d-5c24-4662-9417-0169d9db780e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd13f20e8d22b4f798cef08b52ff1c3ec'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:59:53.424047 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-a412b72d-5c24-4662-9417-0169d9db780e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd13f20e8d22b4f798cef08b52ff1c3ec'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:59:53.439573 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-a412b72d-5c24-4662-9417-0169d9db780e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd13f20e8d22b4f798cef08b52ff1c3ec'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:59:53.440292 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-a412b72d-5c24-4662-9417-0169d9db780e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd13f20e8d22b4f798cef08b52ff1c3ec'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:59:53.530390 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1186/2386] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 22:59:52 2022] DELETE /identity/v3/users/d13f20e8d22b4f798cef08b52ff1c3ec => generated 0 bytes in 658 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:59:54.002823 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b439b8-87ec-40b4-bb4e-8b14a37ff525 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:54.004488 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-59b439b8-87ec-40b4-bb4e-8b14a37ff525 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:59:54.030270 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b439b8-87ec-40b4-bb4e-8b14a37ff525 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:54.064100 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:59:54.135355 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e48c3f-e742-435c-b7f3-bee9f51a3f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:54.138993 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e48c3f-e742-435c-b7f3-bee9f51a3f1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:54.139396 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e48c3f-e742-435c-b7f3-bee9f51a3f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:54.139782 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e48c3f-e742-435c-b7f3-bee9f51a3f1a admin admin] PATH_INFO: `/v3/projects/223f259dd9ad4f41b3212dad93b8bf29` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:54.158349 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e48c3f-e742-435c-b7f3-bee9f51a3f1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=223f259dd9ad4f41b3212dad93b8bf29)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:54.161100 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e48c3f-e742-435c-b7f3-bee9f51a3f1a admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:54.310826 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-a4e48c3f-e742-435c-b7f3-bee9f51a3f1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '223f259dd9ad4f41b3212dad93b8bf29', 'request_id': 'req-a4e48c3f-e742-435c-b7f3-bee9f51a3f1a'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:59:54.314641 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-a4e48c3f-e742-435c-b7f3-bee9f51a3f1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '223f259dd9ad4f41b3212dad93b8bf29', 'request_id': 'req-a4e48c3f-e742-435c-b7f3-bee9f51a3f1a'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:59:54.401093 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-a4e48c3f-e742-435c-b7f3-bee9f51a3f1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '223f259dd9ad4f41b3212dad93b8bf29', 'request_id': 'req-a4e48c3f-e742-435c-b7f3-bee9f51a3f1a'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:59:54.414462 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-a4e48c3f-e742-435c-b7f3-bee9f51a3f1a admin admin] The token cache is being invalidate because project 223f259dd9ad4f41b3212dad93b8bf29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 22:59:54.414751 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-a4e48c3f-e742-435c-b7f3-bee9f51a3f1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:59:54.415421 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-a4e48c3f-e742-435c-b7f3-bee9f51a3f1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 22:59:54.423014 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ec7fa5-7d06-4296-a599-64e9ef0b24c7 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:54.423625 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ec7fa5-7d06-4296-a599-64e9ef0b24c7 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:54.423625 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ec7fa5-7d06-4296-a599-64e9ef0b24c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:54.426640 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1201/2387] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 22:59:54 2022] DELETE /identity/v3/projects/223f259dd9ad4f41b3212dad93b8bf29 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 22:59:54.442496 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-15ec7fa5-7d06-4296-a599-64e9ef0b24c7 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:59:54.443752 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-15ec7fa5-7d06-4296-a599-64e9ef0b24c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:59:54.513808 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1187/2388] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:59:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:59:54.522520 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1045e-792e-45cc-8ad0-6c0e20835ce9 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:54.522871 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1045e-792e-45cc-8ad0-6c0e20835ce9 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:54.523169 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1045e-792e-45cc-8ad0-6c0e20835ce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:54.542874 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-3ad1045e-792e-45cc-8ad0-6c0e20835ce9 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:59:54.544328 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-3ad1045e-792e-45cc-8ad0-6c0e20835ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:59:54.553032 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1202/2389] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:59:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:59:54.557273 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ec7fa5-7d06-4296-a599-64e9ef0b24c7 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:54.568471 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ec7fa5-7d06-4296-a599-64e9ef0b24c7 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:54.573166 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0740db6-36d0-4136-9648-b0f1d4eab6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:54.573822 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0740db6-36d0-4136-9648-b0f1d4eab6af admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:54.574102 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0740db6-36d0-4136-9648-b0f1d4eab6af admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:54.574374 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0740db6-36d0-4136-9648-b0f1d4eab6af admin admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:54.576878 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0740db6-36d0-4136-9648-b0f1d4eab6af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 22:59:54.576878 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0740db6-36d0-4136-9648-b0f1d4eab6af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:54.577104 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0740db6-36d0-4136-9648-b0f1d4eab6af admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:54.589669 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1188/2390] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 22:59:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 22:59:54.594556 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad1045e-792e-45cc-8ad0-6c0e20835ce9 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:54.605908 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad1045e-792e-45cc-8ad0-6c0e20835ce9 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:54.610699 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c55caa1-9997-46e7-b6c8-79d83d2c8791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:54.611385 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55caa1-9997-46e7-b6c8-79d83d2c8791 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:54.611669 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55caa1-9997-46e7-b6c8-79d83d2c8791 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:54.611944 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55caa1-9997-46e7-b6c8-79d83d2c8791 admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:54.612627 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c55caa1-9997-46e7-b6c8-79d83d2c8791 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:54.614619 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c55caa1-9997-46e7-b6c8-79d83d2c8791 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:54.702089 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1203/2391] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 22:59:54 2022] POST /identity/v3/projects => generated 367 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 22:59:54.706153 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0740db6-36d0-4136-9648-b0f1d4eab6af admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:54.718435 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0740db6-36d0-4136-9648-b0f1d4eab6af admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:54.790913 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf7060d-9584-4590-89a4-e38e33301b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:54.791516 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf7060d-9584-4590-89a4-e38e33301b8b admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:54.791802 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf7060d-9584-4590-89a4-e38e33301b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:54.792080 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf7060d-9584-4590-89a4-e38e33301b8b admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:54.792771 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf7060d-9584-4590-89a4-e38e33301b8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:54.794691 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf7060d-9584-4590-89a4-e38e33301b8b admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:54.915963 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1189/2392] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 22:59:54 2022] POST /identity/v3/users => generated 284 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 22:59:54.919771 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c55caa1-9997-46e7-b6c8-79d83d2c8791 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:54.931445 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c55caa1-9997-46e7-b6c8-79d83d2c8791 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:54.939317 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f456a0d8-d10f-4755-8a48-76b1cd1f6fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:54.940122 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f456a0d8-d10f-4755-8a48-76b1cd1f6fdc admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:54.940507 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f456a0d8-d10f-4755-8a48-76b1cd1f6fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:54.941485 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f456a0d8-d10f-4755-8a48-76b1cd1f6fdc admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:54.942328 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f456a0d8-d10f-4755-8a48-76b1cd1f6fdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:54.944161 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f456a0d8-d10f-4755-8a48-76b1cd1f6fdc admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:54.960676 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1204/2393] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:59:54 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 22:59:54.961912 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf7060d-9584-4590-89a4-e38e33301b8b admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:54.975207 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf7060d-9584-4590-89a4-e38e33301b8b admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:54.980294 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0793a8a-3568-4b3a-a8b3-ccd665912bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:54.981093 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0793a8a-3568-4b3a-a8b3-ccd665912bef admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:54.981457 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0793a8a-3568-4b3a-a8b3-ccd665912bef admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:54.981738 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0793a8a-3568-4b3a-a8b3-ccd665912bef admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:54.982472 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0793a8a-3568-4b3a-a8b3-ccd665912bef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:54.984237 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0793a8a-3568-4b3a-a8b3-ccd665912bef admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:54.995331 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:59:55.001907 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1190/2394] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 22:59:54 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 22:59:55.006764 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f456a0d8-d10f-4755-8a48-76b1cd1f6fdc admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:55.019295 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f456a0d8-d10f-4755-8a48-76b1cd1f6fdc admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:55.024842 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d02957-8083-4236-a446-885c52726d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:55.025785 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d02957-8083-4236-a446-885c52726d4a admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:55.026075 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d02957-8083-4236-a446-885c52726d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:55.026359 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d02957-8083-4236-a446-885c52726d4a admin admin] PATH_INFO: `/v3/projects/3429139a6e494a7290d91dd3cb8f712c/users/932c6d0ae774493a852781e4c8d83e67/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:55.046224 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d02957-8083-4236-a446-885c52726d4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3429139a6e494a7290d91dd3cb8f712c, user_id=932c6d0ae774493a852781e4c8d83e67, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:55.046224 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d02957-8083-4236-a446-885c52726d4a admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:55.119036 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1205/2395] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 22:59:55 2022] PUT /identity/v3/projects/3429139a6e494a7290d91dd3cb8f712c/users/932c6d0ae774493a852781e4c8d83e67/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 22:59:55.124333 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed1a5dd-800d-4f3a-90c2-761ca9f2b1c1 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:55.126564 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed1a5dd-800d-4f3a-90c2-761ca9f2b1c1 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:55.126564 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed1a5dd-800d-4f3a-90c2-761ca9f2b1c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:55.144724 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-9ed1a5dd-800d-4f3a-90c2-761ca9f2b1c1 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:59:55.145956 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-9ed1a5dd-800d-4f3a-90c2-761ca9f2b1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:59:55.230183 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1191/2396] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:59:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:59:55.237004 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d02957-8083-4236-a446-885c52726d4a admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:55.240362 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-55d02957-8083-4236-a446-885c52726d4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:59:55.318740 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 22:59:55.324678 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d02957-8083-4236-a446-885c52726d4a admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:55.329105 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9719b767-3345-465a-890e-3f265d96a622 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:55.329901 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9719b767-3345-465a-890e-3f265d96a622 service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:55.330296 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9719b767-3345-465a-890e-3f265d96a622 service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:55.330827 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9719b767-3345-465a-890e-3f265d96a622 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:55.343213 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9719b767-3345-465a-890e-3f265d96a622 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:55.344748 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9719b767-3345-465a-890e-3f265d96a622 service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:55.363153 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1206/2397] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:59:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:59:56.252265 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed1a5dd-800d-4f3a-90c2-761ca9f2b1c1 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:56.256467 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-9ed1a5dd-800d-4f3a-90c2-761ca9f2b1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:59:56.349637 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed1a5dd-800d-4f3a-90c2-761ca9f2b1c1 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:56.354820 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abafd070-423b-41ca-ac9b-589afbf23e95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:56.355474 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abafd070-423b-41ca-ac9b-589afbf23e95 service cinder] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:56.355862 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abafd070-423b-41ca-ac9b-589afbf23e95 service cinder] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:56.356275 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abafd070-423b-41ca-ac9b-589afbf23e95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:56.358310 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-abafd070-423b-41ca-ac9b-589afbf23e95 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:59:56.440362 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abafd070-423b-41ca-ac9b-589afbf23e95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:56.440362 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abafd070-423b-41ca-ac9b-589afbf23e95 service cinder] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:56.440362 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1192/2398] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:59:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 22:59:57.005936 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98510b0e-1324-4eda-8940-b9493685f535 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:57.006951 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98510b0e-1324-4eda-8940-b9493685f535 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:57.007471 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98510b0e-1324-4eda-8940-b9493685f535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:57.038694 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-98510b0e-1324-4eda-8940-b9493685f535 None None] MFA Rules not processed for user `932c6d0ae774493a852781e4c8d83e67`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:59:57.040756 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-98510b0e-1324-4eda-8940-b9493685f535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:59:57.182264 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1207/2399] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:59:57 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 22:59:57.194500 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abafd070-423b-41ca-ac9b-589afbf23e95 service cinder] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 22:59:57.197679 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-abafd070-423b-41ca-ac9b-589afbf23e95 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:59:57.214105 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abafd070-423b-41ca-ac9b-589afbf23e95 service cinder] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 22:59:57.224531 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ca770-9f57-470e-85a0-1f313ab846dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 22:59:57.225214 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ca770-9f57-470e-85a0-1f313ab846dd service cinder] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:57.225503 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ca770-9f57-470e-85a0-1f313ab846dd service cinder] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:57.225781 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ca770-9f57-470e-85a0-1f313ab846dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:57.242112 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0ca770-9f57-470e-85a0-1f313ab846dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 22:59:57.243935 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0ca770-9f57-470e-85a0-1f313ab846dd service cinder] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 22:59:57.258695 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1193/2400] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 22:59:57 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 22:59:59.168626 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccae57e-ffc9-4332-b71b-e2e1cfdef0c9 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 22:59:59.169179 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccae57e-ffc9-4332-b71b-e2e1cfdef0c9 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 22:59:59.169179 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccae57e-ffc9-4332-b71b-e2e1cfdef0c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 22:59:59.203868 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-3ccae57e-ffc9-4332-b71b-e2e1cfdef0c9 None None] MFA Rules not processed for user `932c6d0ae774493a852781e4c8d83e67`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 22:59:59.205111 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-3ccae57e-ffc9-4332-b71b-e2e1cfdef0c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 22:59:59.219786 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1208/2401] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 22:59:59 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:00:07.253586 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=81 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:00:09.195264 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=40 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:00:10.146277 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459401d-423f-4217-8532-5f20ad03d092 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:10.147775 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459401d-423f-4217-8532-5f20ad03d092 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:10.147775 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459401d-423f-4217-8532-5f20ad03d092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:10.165105 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-6459401d-423f-4217-8532-5f20ad03d092 None None] MFA Rules not processed for user `63cc904578554aea84ff1917a5ed075e`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:00:10.166327 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-6459401d-423f-4217-8532-5f20ad03d092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:00:10.223240 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1194/2402] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jul 21 23:00:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:00:10.235247 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccae57e-ffc9-4332-b71b-e2e1cfdef0c9 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:00:10.236721 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-3ccae57e-ffc9-4332-b71b-e2e1cfdef0c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:00:10.308209 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccae57e-ffc9-4332-b71b-e2e1cfdef0c9 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:00:10.313281 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee531e84-dbee-4ac2-b247-83f2d0126258 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba7eef4e1b27476ab4016449d9c3a4b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:00:10.313934 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee531e84-dbee-4ac2-b247-83f2d0126258 service swift] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:10.314226 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee531e84-dbee-4ac2-b247-83f2d0126258 service swift] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:10.314503 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee531e84-dbee-4ac2-b247-83f2d0126258 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:10.323334 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee531e84-dbee-4ac2-b247-83f2d0126258 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:00:10.324914 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee531e84-dbee-4ac2-b247-83f2d0126258 service swift] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:00:10.333491 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1209/2403] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jul 21 23:00:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 23:00:20.216160 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=25 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:00:20.329617 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=32 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:00:28.832559 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e2a5d2-b724-4add-89cc-01a7db505adb None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:28.838029 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e2a5d2-b724-4add-89cc-01a7db505adb None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:28.838029 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e2a5d2-b724-4add-89cc-01a7db505adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:28.853335 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-06e2a5d2-b724-4add-89cc-01a7db505adb None None] MFA Rules not processed for user `63cc904578554aea84ff1917a5ed075e`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:00:28.854926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-06e2a5d2-b724-4add-89cc-01a7db505adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:00:28.863164 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1195/2404] 10.222.0.52 () {58 vars in 967 bytes} [Thu Jul 21 23:00:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:00:28.875505 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee531e84-dbee-4ac2-b247-83f2d0126258 service swift] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:00:28.886183 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee531e84-dbee-4ac2-b247-83f2d0126258 service swift] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:00:28.891360 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d9773d-c9c9-4be2-af60-106b6e7d0527 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba7eef4e1b27476ab4016449d9c3a4b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:00:28.891950 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d9773d-c9c9-4be2-af60-106b6e7d0527 service swift] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:28.892234 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d9773d-c9c9-4be2-af60-106b6e7d0527 service swift] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:28.892538 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d9773d-c9c9-4be2-af60-106b6e7d0527 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:28.902193 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d9773d-c9c9-4be2-af60-106b6e7d0527 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:00:28.903820 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d9773d-c9c9-4be2-af60-106b6e7d0527 service swift] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:00:28.913982 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1210/2405] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jul 21 23:00:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 21 23:00:32.623032 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440bc031-5555-4fc7-8e24-a591b8ee7892 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:32.624225 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440bc031-5555-4fc7-8e24-a591b8ee7892 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:32.624553 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440bc031-5555-4fc7-8e24-a591b8ee7892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:32.647806 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-440bc031-5555-4fc7-8e24-a591b8ee7892 None None] MFA Rules not processed for user `932c6d0ae774493a852781e4c8d83e67`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:00:32.649402 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-440bc031-5555-4fc7-8e24-a591b8ee7892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:00:32.656801 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1196/2406] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:00:32 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:00:33.560581 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76a260-4bc8-494c-80ee-f35bcabf1450 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:33.560992 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76a260-4bc8-494c-80ee-f35bcabf1450 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:33.561159 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76a260-4bc8-494c-80ee-f35bcabf1450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:33.582894 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-af76a260-4bc8-494c-80ee-f35bcabf1450 None None] MFA Rules not processed for user `63cc904578554aea84ff1917a5ed075e`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:00:33.583234 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-af76a260-4bc8-494c-80ee-f35bcabf1450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:00:33.590439 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1211/2407] 10.222.0.52 () {58 vars in 967 bytes} [Thu Jul 21 23:00:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:00:33.603170 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440bc031-5555-4fc7-8e24-a591b8ee7892 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:00:33.615371 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440bc031-5555-4fc7-8e24-a591b8ee7892 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:00:33.620553 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d5957f-96a8-4d94-a166-53457f03bc34 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba7eef4e1b27476ab4016449d9c3a4b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:00:33.621322 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5957f-96a8-4d94-a166-53457f03bc34 service swift] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:33.621691 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5957f-96a8-4d94-a166-53457f03bc34 service swift] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:33.622034 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5957f-96a8-4d94-a166-53457f03bc34 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:33.631154 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d5957f-96a8-4d94-a166-53457f03bc34 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:00:33.634317 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d5957f-96a8-4d94-a166-53457f03bc34 service swift] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:00:33.641630 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1197/2408] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jul 21 23:00:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 23:00:35.897744 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5919f54c-42fe-4303-9a94-325304febbfe None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:35.898205 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5919f54c-42fe-4303-9a94-325304febbfe None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:35.898322 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5919f54c-42fe-4303-9a94-325304febbfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:35.919540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-5919f54c-42fe-4303-9a94-325304febbfe None None] MFA Rules not processed for user `8c11dc2f4c764dc58458d9226efc878e`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:00:35.921015 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-5919f54c-42fe-4303-9a94-325304febbfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:00:35.929237 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1212/2409] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:00:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:00:39.568487 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d5957f-96a8-4d94-a166-53457f03bc34 service swift] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:00:39.578585 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-03d5957f-96a8-4d94-a166-53457f03bc34 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:00:39.613440 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d5957f-96a8-4d94-a166-53457f03bc34 service swift] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:00:39.616391 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1819e94-355e-4357-9250-cff836d406c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:00:39.617386 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1819e94-355e-4357-9250-cff836d406c5 service neutron] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:39.617832 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1819e94-355e-4357-9250-cff836d406c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:39.618279 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1819e94-355e-4357-9250-cff836d406c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:39.620821 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-f1819e94-355e-4357-9250-cff836d406c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:00:39.712075 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1819e94-355e-4357-9250-cff836d406c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:00:39.713671 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1819e94-355e-4357-9250-cff836d406c5 service neutron] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:00:39.727729 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1198/2410] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:00:39 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 23:00:41.536743 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b5c905-5c6c-4f97-a57c-782c73ca5eb0 None None] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:41.537122 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b5c905-5c6c-4f97-a57c-782c73ca5eb0 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:41.537412 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b5c905-5c6c-4f97-a57c-782c73ca5eb0 None None] PATH_INFO: `/` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:41.539741 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1213/2411] 10.222.0.52 () {54 vars in 921 bytes} [Thu Jul 21 23:00:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 23:00:41.552550 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea9be87-320e-4977-9fd7-f5b5459dce87 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:41.553852 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea9be87-320e-4977-9fd7-f5b5459dce87 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:41.554367 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea9be87-320e-4977-9fd7-f5b5459dce87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:41.611254 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-aea9be87-320e-4977-9fd7-f5b5459dce87 None None] MFA Rules not processed for user `10c0bed169344a77bfad29624c5e5a8e`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:00:41.612209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-aea9be87-320e-4977-9fd7-f5b5459dce87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:00:41.718036 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1199/2412] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Jul 21 23:00:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:00:41.732404 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b5c905-5c6c-4f97-a57c-782c73ca5eb0 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:00:41.734158 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-19b5c905-5c6c-4f97-a57c-782c73ca5eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:00:41.773017 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b5c905-5c6c-4f97-a57c-782c73ca5eb0 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:00:41.781905 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f5a446-7082-406c-be7c-23ce4196a498 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10c0bed169344a77bfad29624c5e5a8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:00:41.783122 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f5a446-7082-406c-be7c-23ce4196a498 service placement] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:41.783714 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f5a446-7082-406c-be7c-23ce4196a498 service placement] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:41.784339 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f5a446-7082-406c-be7c-23ce4196a498 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:41.796814 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f5a446-7082-406c-be7c-23ce4196a498 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:00:41.800111 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f5a446-7082-406c-be7c-23ce4196a498 service placement] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:00:41.851794 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1214/2413] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:00:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:00:44.571231 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea9be87-320e-4977-9fd7-f5b5459dce87 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:00:44.580832 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea9be87-320e-4977-9fd7-f5b5459dce87 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:00:44.585156 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e9ac0c-ad08-45f2-8c8b-143dd86f4d02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:00:44.586001 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e9ac0c-ad08-45f2-8c8b-143dd86f4d02 service neutron] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:44.586287 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e9ac0c-ad08-45f2-8c8b-143dd86f4d02 service neutron] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:44.586566 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e9ac0c-ad08-45f2-8c8b-143dd86f4d02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:44.588498 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-63e9ac0c-ad08-45f2-8c8b-143dd86f4d02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:00:44.602381 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e9ac0c-ad08-45f2-8c8b-143dd86f4d02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:00:44.604278 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e9ac0c-ad08-45f2-8c8b-143dd86f4d02 service neutron] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:00:44.615763 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1200/2414] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:00:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:00:47.453659 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f5a446-7082-406c-be7c-23ce4196a498 service placement] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:00:47.455879 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-f4f5a446-7082-406c-be7c-23ce4196a498 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:00:47.479894 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f5a446-7082-406c-be7c-23ce4196a498 service placement] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:00:47.495961 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4388fa3-5c86-4bf3-93fe-32eed298024d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10c0bed169344a77bfad29624c5e5a8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:00:47.496898 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4388fa3-5c86-4bf3-93fe-32eed298024d service placement] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:47.497347 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4388fa3-5c86-4bf3-93fe-32eed298024d service placement] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:47.498083 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4388fa3-5c86-4bf3-93fe-32eed298024d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:47.503881 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-e4388fa3-5c86-4bf3-93fe-32eed298024d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:00:47.533448 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4388fa3-5c86-4bf3-93fe-32eed298024d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:00:47.537549 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4388fa3-5c86-4bf3-93fe-32eed298024d service placement] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:00:47.556230 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1215/2415] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:00:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:00:47.658934 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e9ac0c-ad08-45f2-8c8b-143dd86f4d02 service neutron] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:00:47.660399 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-63e9ac0c-ad08-45f2-8c8b-143dd86f4d02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:00:47.691941 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e9ac0c-ad08-45f2-8c8b-143dd86f4d02 service neutron] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:00:47.706457 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4388fa3-5c86-4bf3-93fe-32eed298024d service placement] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:00:47.709641 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefa3822-004f-4a71-9fe8-9370a2aa0df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:00:47.711548 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa3822-004f-4a71-9fe8-9370a2aa0df3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:47.712732 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa3822-004f-4a71-9fe8-9370a2aa0df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:47.713101 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa3822-004f-4a71-9fe8-9370a2aa0df3 admin admin] PATH_INFO: `/v3/users/257adcc88884437eb740f78dbd9e19c2` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:47.714565 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefa3822-004f-4a71-9fe8-9370a2aa0df3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=257adcc88884437eb740f78dbd9e19c2)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:00:47.717986 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefa3822-004f-4a71-9fe8-9370a2aa0df3 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:00:47.720307 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4388fa3-5c86-4bf3-93fe-32eed298024d service placement] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:00:47.725800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a51062b-a42a-4ac1-8f74-2c552c5395c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:00:47.726625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51062b-a42a-4ac1-8f74-2c552c5395c8 service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:47.727115 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51062b-a42a-4ac1-8f74-2c552c5395c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:47.727798 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51062b-a42a-4ac1-8f74-2c552c5395c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:47.729864 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-7a51062b-a42a-4ac1-8f74-2c552c5395c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:00:47.746925 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a51062b-a42a-4ac1-8f74-2c552c5395c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:00:47.748604 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a51062b-a42a-4ac1-8f74-2c552c5395c8 service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:00:47.760783 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1216/2416] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:00:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 23:00:47.939836 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-eefa3822-004f-4a71-9fe8-9370a2aa0df3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '257adcc88884437eb740f78dbd9e19c2'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:47.945432 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:00:47.960721 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-eefa3822-004f-4a71-9fe8-9370a2aa0df3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '257adcc88884437eb740f78dbd9e19c2'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:47.961474 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-eefa3822-004f-4a71-9fe8-9370a2aa0df3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '257adcc88884437eb740f78dbd9e19c2'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:47.984625 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-eefa3822-004f-4a71-9fe8-9370a2aa0df3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '257adcc88884437eb740f78dbd9e19c2'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:47.985365 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-eefa3822-004f-4a71-9fe8-9370a2aa0df3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '257adcc88884437eb740f78dbd9e19c2'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:48.079057 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1201/2417] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 23:00:47 2022] DELETE /identity/v3/users/257adcc88884437eb740f78dbd9e19c2 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:00:48.745828 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a51062b-a42a-4ac1-8f74-2c552c5395c8 service neutron] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:00:48.756140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a51062b-a42a-4ac1-8f74-2c552c5395c8 service neutron] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:00:48.870123 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a423a355-3ec0-46ca-b743-00fea7edba59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:00:48.871583 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a423a355-3ec0-46ca-b743-00fea7edba59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:48.871583 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a423a355-3ec0-46ca-b743-00fea7edba59 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:48.871784 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a423a355-3ec0-46ca-b743-00fea7edba59 admin admin] PATH_INFO: `/v3/projects/5dbf5236757941d6b4db3517e1757621` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:48.874857 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a423a355-3ec0-46ca-b743-00fea7edba59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5dbf5236757941d6b4db3517e1757621)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:00:48.876850 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a423a355-3ec0-46ca-b743-00fea7edba59 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:00:49.047532 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-a423a355-3ec0-46ca-b743-00fea7edba59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5dbf5236757941d6b4db3517e1757621', 'request_id': 'req-a423a355-3ec0-46ca-b743-00fea7edba59'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:49.054665 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-a423a355-3ec0-46ca-b743-00fea7edba59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5dbf5236757941d6b4db3517e1757621', 'request_id': 'req-a423a355-3ec0-46ca-b743-00fea7edba59'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:49.127838 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-a423a355-3ec0-46ca-b743-00fea7edba59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5dbf5236757941d6b4db3517e1757621', 'request_id': 'req-a423a355-3ec0-46ca-b743-00fea7edba59'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:49.162133 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-a423a355-3ec0-46ca-b743-00fea7edba59 admin admin] The token cache is being invalidate because project 5dbf5236757941d6b4db3517e1757621 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:00:49.162806 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-a423a355-3ec0-46ca-b743-00fea7edba59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:49.164168 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-a423a355-3ec0-46ca-b743-00fea7edba59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:49.175147 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1217/2418] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 23:00:48 2022] DELETE /identity/v3/projects/5dbf5236757941d6b4db3517e1757621 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:00:49.176454 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefa3822-004f-4a71-9fe8-9370a2aa0df3 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:00:49.178445 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-eefa3822-004f-4a71-9fe8-9370a2aa0df3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:00:49.193745 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefa3822-004f-4a71-9fe8-9370a2aa0df3 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:00:49.289119 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87d02be-1195-46fa-bd18-864bd84d2bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:00:49.290472 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d02be-1195-46fa-bd18-864bd84d2bf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:49.291042 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d02be-1195-46fa-bd18-864bd84d2bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:49.291588 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d02be-1195-46fa-bd18-864bd84d2bf9 admin admin] PATH_INFO: `/v3/users/4d7b543f09474961938ddb1585bd0a38` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:49.293138 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87d02be-1195-46fa-bd18-864bd84d2bf9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d7b543f09474961938ddb1585bd0a38)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:00:49.295396 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87d02be-1195-46fa-bd18-864bd84d2bf9 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:00:49.536570 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-e87d02be-1195-46fa-bd18-864bd84d2bf9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d7b543f09474961938ddb1585bd0a38'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:49.565519 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-e87d02be-1195-46fa-bd18-864bd84d2bf9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d7b543f09474961938ddb1585bd0a38'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:49.567183 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-e87d02be-1195-46fa-bd18-864bd84d2bf9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d7b543f09474961938ddb1585bd0a38'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:49.587694 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-e87d02be-1195-46fa-bd18-864bd84d2bf9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d7b543f09474961938ddb1585bd0a38'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:49.594286 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-e87d02be-1195-46fa-bd18-864bd84d2bf9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d7b543f09474961938ddb1585bd0a38'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:49.684195 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1202/2419] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 23:00:49 2022] DELETE /identity/v3/users/4d7b543f09474961938ddb1585bd0a38 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:00:49.944575 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a423a355-3ec0-46ca-b743-00fea7edba59 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:00:49.946840 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-a423a355-3ec0-46ca-b743-00fea7edba59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:00:49.979087 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a423a355-3ec0-46ca-b743-00fea7edba59 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:00:50.058080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:00:50.107902 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa43bfcc-6463-4a78-8308-8d8ec59a149c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:00:50.109123 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43bfcc-6463-4a78-8308-8d8ec59a149c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:50.109709 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43bfcc-6463-4a78-8308-8d8ec59a149c admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:50.110151 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43bfcc-6463-4a78-8308-8d8ec59a149c admin admin] PATH_INFO: `/v3/projects/6a7d54c1b920447f9bab1caf2e579f79` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:50.111612 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa43bfcc-6463-4a78-8308-8d8ec59a149c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a7d54c1b920447f9bab1caf2e579f79)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:00:50.114198 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa43bfcc-6463-4a78-8308-8d8ec59a149c admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:00:50.271606 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-fa43bfcc-6463-4a78-8308-8d8ec59a149c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a7d54c1b920447f9bab1caf2e579f79', 'request_id': 'req-fa43bfcc-6463-4a78-8308-8d8ec59a149c'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:50.278500 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-fa43bfcc-6463-4a78-8308-8d8ec59a149c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a7d54c1b920447f9bab1caf2e579f79', 'request_id': 'req-fa43bfcc-6463-4a78-8308-8d8ec59a149c'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:50.375157 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-fa43bfcc-6463-4a78-8308-8d8ec59a149c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a7d54c1b920447f9bab1caf2e579f79', 'request_id': 'req-fa43bfcc-6463-4a78-8308-8d8ec59a149c'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:50.400557 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-fa43bfcc-6463-4a78-8308-8d8ec59a149c admin admin] The token cache is being invalidate because project 6a7d54c1b920447f9bab1caf2e579f79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:00:50.401216 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-fa43bfcc-6463-4a78-8308-8d8ec59a149c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:50.402086 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-fa43bfcc-6463-4a78-8308-8d8ec59a149c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:50.411547 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1218/2420] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 23:00:49 2022] DELETE /identity/v3/projects/6a7d54c1b920447f9bab1caf2e579f79 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:00:50.818611 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87d02be-1195-46fa-bd18-864bd84d2bf9 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:00:50.820149 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-e87d02be-1195-46fa-bd18-864bd84d2bf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:00:50.842408 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87d02be-1195-46fa-bd18-864bd84d2bf9 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:00:50.900648 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:00:50.914364 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e69727-1f94-4231-bef0-75fa38897274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:00:50.915477 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e69727-1f94-4231-bef0-75fa38897274 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:50.915967 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e69727-1f94-4231-bef0-75fa38897274 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:50.916463 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e69727-1f94-4231-bef0-75fa38897274 admin admin] PATH_INFO: `/v3/users/ef9ff560b7044ea2a18b3c3f8295937e` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:50.927249 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e69727-1f94-4231-bef0-75fa38897274 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef9ff560b7044ea2a18b3c3f8295937e)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:00:50.927249 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e69727-1f94-4231-bef0-75fa38897274 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:00:51.125054 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-71e69727-1f94-4231-bef0-75fa38897274 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef9ff560b7044ea2a18b3c3f8295937e'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:51.138414 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-71e69727-1f94-4231-bef0-75fa38897274 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef9ff560b7044ea2a18b3c3f8295937e'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:51.139405 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-71e69727-1f94-4231-bef0-75fa38897274 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef9ff560b7044ea2a18b3c3f8295937e'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:51.153396 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-71e69727-1f94-4231-bef0-75fa38897274 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef9ff560b7044ea2a18b3c3f8295937e'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:51.154557 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-71e69727-1f94-4231-bef0-75fa38897274 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef9ff560b7044ea2a18b3c3f8295937e'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:51.286611 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1203/2421] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 23:00:50 2022] DELETE /identity/v3/users/ef9ff560b7044ea2a18b3c3f8295937e => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:00:51.545405 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa43bfcc-6463-4a78-8308-8d8ec59a149c admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:00:51.547110 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-fa43bfcc-6463-4a78-8308-8d8ec59a149c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:00:51.562965 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa43bfcc-6463-4a78-8308-8d8ec59a149c admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:00:51.662012 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fca5cf1-eeb7-4535-9451-19ae351637ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:00:51.662978 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fca5cf1-eeb7-4535-9451-19ae351637ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:51.663263 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fca5cf1-eeb7-4535-9451-19ae351637ce admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:51.663537 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fca5cf1-eeb7-4535-9451-19ae351637ce admin admin] PATH_INFO: `/v3/projects/08576395a1f1437385d2f24b482572a6` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:51.664985 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fca5cf1-eeb7-4535-9451-19ae351637ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=08576395a1f1437385d2f24b482572a6)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:00:51.667121 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fca5cf1-eeb7-4535-9451-19ae351637ce admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:00:51.778367 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-7fca5cf1-eeb7-4535-9451-19ae351637ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08576395a1f1437385d2f24b482572a6', 'request_id': 'req-7fca5cf1-eeb7-4535-9451-19ae351637ce'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:51.782917 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:00:51.784087 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-7fca5cf1-eeb7-4535-9451-19ae351637ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08576395a1f1437385d2f24b482572a6', 'request_id': 'req-7fca5cf1-eeb7-4535-9451-19ae351637ce'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:51.870321 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-7fca5cf1-eeb7-4535-9451-19ae351637ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08576395a1f1437385d2f24b482572a6', 'request_id': 'req-7fca5cf1-eeb7-4535-9451-19ae351637ce'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:51.889079 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-7fca5cf1-eeb7-4535-9451-19ae351637ce admin admin] The token cache is being invalidate because project 08576395a1f1437385d2f24b482572a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:00:51.889389 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-7fca5cf1-eeb7-4535-9451-19ae351637ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:51.894107 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-7fca5cf1-eeb7-4535-9451-19ae351637ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:51.895869 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1219/2422] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 23:00:51 2022] DELETE /identity/v3/projects/08576395a1f1437385d2f24b482572a6 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:00:51.900905 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3ad76-3b14-4b59-b541-6a18a4113c35 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:51.901751 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3ad76-3b14-4b59-b541-6a18a4113c35 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:51.902039 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3ad76-3b14-4b59-b541-6a18a4113c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:51.923550 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-a3f3ad76-3b14-4b59-b541-6a18a4113c35 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:00:51.925246 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-a3f3ad76-3b14-4b59-b541-6a18a4113c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:00:51.992844 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1204/2423] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:00:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 23:00:52.001250 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7201b0-14f6-4320-92c7-fca0fa41f317 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:52.001572 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7201b0-14f6-4320-92c7-fca0fa41f317 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:52.001855 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7201b0-14f6-4320-92c7-fca0fa41f317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:52.031254 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-bd7201b0-14f6-4320-92c7-fca0fa41f317 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:00:52.032576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-bd7201b0-14f6-4320-92c7-fca0fa41f317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:00:52.041598 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1220/2424] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:00:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:00:52.046092 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f3ad76-3b14-4b59-b541-6a18a4113c35 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:00:52.063850 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f3ad76-3b14-4b59-b541-6a18a4113c35 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:00:52.077442 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39af8148-6450-48c1-aa88-4ff499fd0edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:00:52.077442 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af8148-6450-48c1-aa88-4ff499fd0edb admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:52.077442 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af8148-6450-48c1-aa88-4ff499fd0edb admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:52.077442 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af8148-6450-48c1-aa88-4ff499fd0edb admin admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:52.078161 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39af8148-6450-48c1-aa88-4ff499fd0edb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 23:00:52.078161 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39af8148-6450-48c1-aa88-4ff499fd0edb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:00:52.078161 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39af8148-6450-48c1-aa88-4ff499fd0edb admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:00:52.092161 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1205/2425] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 23:00:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 23:00:52.106692 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7201b0-14f6-4320-92c7-fca0fa41f317 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:00:52.117131 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7201b0-14f6-4320-92c7-fca0fa41f317 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:00:52.123734 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3280b01b-b46c-4f59-a158-cb0cd4da92a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:00:52.124425 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3280b01b-b46c-4f59-a158-cb0cd4da92a7 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:52.125368 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3280b01b-b46c-4f59-a158-cb0cd4da92a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:52.125368 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3280b01b-b46c-4f59-a158-cb0cd4da92a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:52.125671 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3280b01b-b46c-4f59-a158-cb0cd4da92a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:00:52.127659 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3280b01b-b46c-4f59-a158-cb0cd4da92a7 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:00:52.209728 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1221/2426] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 23:00:52 2022] POST /identity/v3/projects => generated 381 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:00:52.214532 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39af8148-6450-48c1-aa88-4ff499fd0edb admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:00:52.225612 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39af8148-6450-48c1-aa88-4ff499fd0edb admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:00:52.295309 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:00:52.306955 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f6938d-d7c6-4302-94eb-6bbd4914a9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:00:52.307798 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6938d-d7c6-4302-94eb-6bbd4914a9d0 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:52.309699 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6938d-d7c6-4302-94eb-6bbd4914a9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:52.309699 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6938d-d7c6-4302-94eb-6bbd4914a9d0 admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:52.309699 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f6938d-d7c6-4302-94eb-6bbd4914a9d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:00:52.311589 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f6938d-d7c6-4302-94eb-6bbd4914a9d0 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:00:52.462071 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1206/2427] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 23:00:52 2022] POST /identity/v3/users => generated 291 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 23:00:52.484378 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3280b01b-b46c-4f59-a158-cb0cd4da92a7 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:00:52.497286 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3280b01b-b46c-4f59-a158-cb0cd4da92a7 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:00:52.503007 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877f2ab7-bb40-4b60-89f6-8f2ac0c97edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:00:52.503693 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f2ab7-bb40-4b60-89f6-8f2ac0c97edf admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:52.512421 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f2ab7-bb40-4b60-89f6-8f2ac0c97edf admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:52.512421 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f2ab7-bb40-4b60-89f6-8f2ac0c97edf admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:52.512421 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877f2ab7-bb40-4b60-89f6-8f2ac0c97edf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:00:52.512421 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877f2ab7-bb40-4b60-89f6-8f2ac0c97edf admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:00:52.520683 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1222/2428] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:00:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:00:52.521961 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f6938d-d7c6-4302-94eb-6bbd4914a9d0 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:00:52.532691 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f6938d-d7c6-4302-94eb-6bbd4914a9d0 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:00:52.538620 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e10013-74e3-4b38-8a32-24365af771bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:00:52.543257 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e10013-74e3-4b38-8a32-24365af771bd admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:52.543257 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e10013-74e3-4b38-8a32-24365af771bd admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:52.543257 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e10013-74e3-4b38-8a32-24365af771bd admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:52.543257 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e10013-74e3-4b38-8a32-24365af771bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:00:52.543257 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e10013-74e3-4b38-8a32-24365af771bd admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:00:52.553453 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1207/2429] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:00:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:00:52.557822 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877f2ab7-bb40-4b60-89f6-8f2ac0c97edf admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:00:52.568513 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877f2ab7-bb40-4b60-89f6-8f2ac0c97edf admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:00:52.573637 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568da580-daff-4628-b26d-c974add707b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:00:52.575492 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568da580-daff-4628-b26d-c974add707b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:52.575492 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568da580-daff-4628-b26d-c974add707b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:52.575492 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568da580-daff-4628-b26d-c974add707b6 admin admin] PATH_INFO: `/v3/projects/5f563abb69484f488736e9bb3dc659a8/users/e6ba879c120042aab28442b199537e99/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:52.594868 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568da580-daff-4628-b26d-c974add707b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f563abb69484f488736e9bb3dc659a8, user_id=e6ba879c120042aab28442b199537e99, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:00:52.598082 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568da580-daff-4628-b26d-c974add707b6 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:00:52.699602 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1223/2430] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 23:00:52 2022] PUT /identity/v3/projects/5f563abb69484f488736e9bb3dc659a8/users/e6ba879c120042aab28442b199537e99/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:00:52.709669 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d2cff-66df-4ea2-8260-ef2b3629e06a None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:52.709958 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d2cff-66df-4ea2-8260-ef2b3629e06a None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:52.713506 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d2cff-66df-4ea2-8260-ef2b3629e06a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:52.738012 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-445d2cff-66df-4ea2-8260-ef2b3629e06a None None] MFA Rules not processed for user `e6ba879c120042aab28442b199537e99`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:00:52.739828 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-445d2cff-66df-4ea2-8260-ef2b3629e06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:00:52.808757 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1208/2431] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:00:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:00:52.813246 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568da580-daff-4628-b26d-c974add707b6 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:00:52.879368 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568da580-daff-4628-b26d-c974add707b6 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:00:52.884495 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1e4312-a464-4698-8411-30b87f7f07c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:00:52.885297 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1e4312-a464-4698-8411-30b87f7f07c2 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:52.885586 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1e4312-a464-4698-8411-30b87f7f07c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:52.885865 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1e4312-a464-4698-8411-30b87f7f07c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:52.886568 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1e4312-a464-4698-8411-30b87f7f07c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:00:52.888552 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1e4312-a464-4698-8411-30b87f7f07c2 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:00:52.973536 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1224/2432] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 23:00:52 2022] POST /identity/v3/projects => generated 385 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:00:52.977918 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d2cff-66df-4ea2-8260-ef2b3629e06a None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:00:52.989345 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d2cff-66df-4ea2-8260-ef2b3629e06a None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:00:53.055615 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd67a9-edf6-4143-a8e0-49cc44156810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:00:53.056534 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd67a9-edf6-4143-a8e0-49cc44156810 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:53.056817 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd67a9-edf6-4143-a8e0-49cc44156810 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:53.057093 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd67a9-edf6-4143-a8e0-49cc44156810 admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:53.057789 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fd67a9-edf6-4143-a8e0-49cc44156810 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:00:53.059782 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fd67a9-edf6-4143-a8e0-49cc44156810 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:00:53.131299 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1209/2433] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 23:00:52 2022] POST /identity/v3/users => generated 299 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 23:00:53.132397 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1e4312-a464-4698-8411-30b87f7f07c2 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:00:53.144621 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1e4312-a464-4698-8411-30b87f7f07c2 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:00:53.155013 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c546d4a9-a449-47ea-bf8f-7cdb5884d5cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:00:53.155898 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c546d4a9-a449-47ea-bf8f-7cdb5884d5cc admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:53.156206 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c546d4a9-a449-47ea-bf8f-7cdb5884d5cc admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:53.156472 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c546d4a9-a449-47ea-bf8f-7cdb5884d5cc admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:53.157245 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c546d4a9-a449-47ea-bf8f-7cdb5884d5cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:00:53.160091 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c546d4a9-a449-47ea-bf8f-7cdb5884d5cc admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:00:53.302062 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1225/2434] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:00:53.308878 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd67a9-edf6-4143-a8e0-49cc44156810 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:00:53.321313 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd67a9-edf6-4143-a8e0-49cc44156810 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:00:53.327282 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c7236-0925-4bf7-98d2-f1500a14a9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:00:53.328256 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c7236-0925-4bf7-98d2-f1500a14a9b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:53.328548 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c7236-0925-4bf7-98d2-f1500a14a9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:53.328830 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c7236-0925-4bf7-98d2-f1500a14a9b8 admin admin] PATH_INFO: `/v3/projects/ed902da7c1c54ef09c1f6914630232ab/users/7766b5d8d7fd48b99dd7fcf8b16562b8/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:53.359229 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074c7236-0925-4bf7-98d2-f1500a14a9b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed902da7c1c54ef09c1f6914630232ab, user_id=7766b5d8d7fd48b99dd7fcf8b16562b8, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:00:53.361698 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074c7236-0925-4bf7-98d2-f1500a14a9b8 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:00:53.435230 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1210/2435] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 23:00:53 2022] PUT /identity/v3/projects/ed902da7c1c54ef09c1f6914630232ab/users/7766b5d8d7fd48b99dd7fcf8b16562b8/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:00:53.439148 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c546d4a9-a449-47ea-bf8f-7cdb5884d5cc admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:00:53.482582 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c546d4a9-a449-47ea-bf8f-7cdb5884d5cc admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:00:53.585507 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:00:53.592549 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937b89b9-6f8c-4332-92ce-365ebd1105ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:00:53.593234 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b89b9-6f8c-4332-92ce-365ebd1105ae admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:53.593535 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b89b9-6f8c-4332-92ce-365ebd1105ae admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:53.593799 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b89b9-6f8c-4332-92ce-365ebd1105ae admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:53.594536 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937b89b9-6f8c-4332-92ce-365ebd1105ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:00:53.599814 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937b89b9-6f8c-4332-92ce-365ebd1105ae admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:00:53.616543 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1226/2436] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:00:53.622645 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c7236-0925-4bf7-98d2-f1500a14a9b8 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:00:53.644077 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c7236-0925-4bf7-98d2-f1500a14a9b8 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:00:53.650172 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97358731-d8c0-45a6-b150-8826f5fd2cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:00:53.651314 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97358731-d8c0-45a6-b150-8826f5fd2cf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:53.651439 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97358731-d8c0-45a6-b150-8826f5fd2cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:53.651650 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97358731-d8c0-45a6-b150-8826f5fd2cf4 admin admin] PATH_INFO: `/v3/projects/ed902da7c1c54ef09c1f6914630232ab/users/7766b5d8d7fd48b99dd7fcf8b16562b8/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:53.653596 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97358731-d8c0-45a6-b150-8826f5fd2cf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed902da7c1c54ef09c1f6914630232ab, user_id=7766b5d8d7fd48b99dd7fcf8b16562b8, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:00:53.656263 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97358731-d8c0-45a6-b150-8826f5fd2cf4 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:00:53.674396 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1211/2437] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 23:00:53 2022] PUT /identity/v3/projects/ed902da7c1c54ef09c1f6914630232ab/users/7766b5d8d7fd48b99dd7fcf8b16562b8/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:00:53.686086 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007abee-b79d-4684-ab09-ccfaa9c33e40 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:53.686325 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007abee-b79d-4684-ab09-ccfaa9c33e40 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:53.686655 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007abee-b79d-4684-ab09-ccfaa9c33e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:53.715032 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-5007abee-b79d-4684-ab09-ccfaa9c33e40 None None] MFA Rules not processed for user `7766b5d8d7fd48b99dd7fcf8b16562b8`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:00:53.716618 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-5007abee-b79d-4684-ab09-ccfaa9c33e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:00:53.794130 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1227/2438] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:00:53 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:00:53.809746 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97358731-d8c0-45a6-b150-8826f5fd2cf4 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:00:53.811722 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-97358731-d8c0-45a6-b150-8826f5fd2cf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:00:53.830435 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:00:53.879877 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97358731-d8c0-45a6-b150-8826f5fd2cf4 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:00:53.885308 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb06fc1-bbe8-41b5-85c4-64fde7cebe4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:00:53.885973 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb06fc1-bbe8-41b5-85c4-64fde7cebe4e service cinder] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:53.886263 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb06fc1-bbe8-41b5-85c4-64fde7cebe4e service cinder] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:53.886690 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb06fc1-bbe8-41b5-85c4-64fde7cebe4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:53.898916 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb06fc1-bbe8-41b5-85c4-64fde7cebe4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:00:53.900374 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb06fc1-bbe8-41b5-85c4-64fde7cebe4e service cinder] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:00:53.910145 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1212/2439] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:00:53 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:00:53.939103 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5007abee-b79d-4684-ab09-ccfaa9c33e40 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:00:54.003321 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5007abee-b79d-4684-ab09-ccfaa9c33e40 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:00:54.007911 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8e67cf-5334-465f-9099-7d78e9398f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:00:54.008809 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e67cf-5334-465f-9099-7d78e9398f2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:54.009125 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e67cf-5334-465f-9099-7d78e9398f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:54.009368 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e67cf-5334-465f-9099-7d78e9398f2c admin admin] PATH_INFO: `/v3/users/e6ba879c120042aab28442b199537e99` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:54.010581 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8e67cf-5334-465f-9099-7d78e9398f2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6ba879c120042aab28442b199537e99)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:00:54.012522 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8e67cf-5334-465f-9099-7d78e9398f2c admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:00:54.198311 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-0f8e67cf-5334-465f-9099-7d78e9398f2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6ba879c120042aab28442b199537e99'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:54.205528 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-0f8e67cf-5334-465f-9099-7d78e9398f2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6ba879c120042aab28442b199537e99'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:54.206185 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-0f8e67cf-5334-465f-9099-7d78e9398f2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6ba879c120042aab28442b199537e99'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:54.222753 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-0f8e67cf-5334-465f-9099-7d78e9398f2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6ba879c120042aab28442b199537e99'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:54.228756 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:00:54.306501 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-0f8e67cf-5334-465f-9099-7d78e9398f2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6ba879c120042aab28442b199537e99'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:54.313329 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1228/2440] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 23:00:53 2022] DELETE /identity/v3/users/e6ba879c120042aab28442b199537e99 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:00:54.318762 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e153e5c-e704-4b77-a74b-d1722c567f2c None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:54.319287 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e153e5c-e704-4b77-a74b-d1722c567f2c None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:54.319388 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e153e5c-e704-4b77-a74b-d1722c567f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:54.340459 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-2e153e5c-e704-4b77-a74b-d1722c567f2c None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:00:54.341675 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-2e153e5c-e704-4b77-a74b-d1722c567f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:00:54.419997 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1213/2441] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:00:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 23:00:54.429749 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8e67cf-5334-465f-9099-7d78e9398f2c admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:00:54.431495 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-0f8e67cf-5334-465f-9099-7d78e9398f2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:00:54.499408 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8e67cf-5334-465f-9099-7d78e9398f2c admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:00:54.503767 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46b4cbc-1add-4b4e-9eba-67f431ecae34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:00:54.504552 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46b4cbc-1add-4b4e-9eba-67f431ecae34 service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:54.505516 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46b4cbc-1add-4b4e-9eba-67f431ecae34 service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:54.505866 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46b4cbc-1add-4b4e-9eba-67f431ecae34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:54.516014 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46b4cbc-1add-4b4e-9eba-67f431ecae34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:00:54.517454 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46b4cbc-1add-4b4e-9eba-67f431ecae34 service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:00:54.527381 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1229/2442] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:00:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:00:55.022613 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e153e5c-e704-4b77-a74b-d1722c567f2c None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:00:55.024783 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-2e153e5c-e704-4b77-a74b-d1722c567f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:00:55.041708 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e153e5c-e704-4b77-a74b-d1722c567f2c None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:00:55.053961 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb0cd35-c743-4f00-96d5-eb7f067b3802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:00:55.054963 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0cd35-c743-4f00-96d5-eb7f067b3802 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:55.055358 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0cd35-c743-4f00-96d5-eb7f067b3802 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:55.056265 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0cd35-c743-4f00-96d5-eb7f067b3802 admin admin] PATH_INFO: `/v3/projects/5f563abb69484f488736e9bb3dc659a8` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:55.057412 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb0cd35-c743-4f00-96d5-eb7f067b3802 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f563abb69484f488736e9bb3dc659a8)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:00:55.059248 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb0cd35-c743-4f00-96d5-eb7f067b3802 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:00:55.229158 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-5eb0cd35-c743-4f00-96d5-eb7f067b3802 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f563abb69484f488736e9bb3dc659a8', 'request_id': 'req-5eb0cd35-c743-4f00-96d5-eb7f067b3802'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:55.236753 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-5eb0cd35-c743-4f00-96d5-eb7f067b3802 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f563abb69484f488736e9bb3dc659a8', 'request_id': 'req-5eb0cd35-c743-4f00-96d5-eb7f067b3802'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:55.356252 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-5eb0cd35-c743-4f00-96d5-eb7f067b3802 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f563abb69484f488736e9bb3dc659a8', 'request_id': 'req-5eb0cd35-c743-4f00-96d5-eb7f067b3802'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:55.377840 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-5eb0cd35-c743-4f00-96d5-eb7f067b3802 admin admin] The token cache is being invalidate because project 5f563abb69484f488736e9bb3dc659a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:00:55.378463 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-5eb0cd35-c743-4f00-96d5-eb7f067b3802 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:55.381356 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-5eb0cd35-c743-4f00-96d5-eb7f067b3802 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:55.382001 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1214/2443] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 23:00:55 2022] DELETE /identity/v3/projects/5f563abb69484f488736e9bb3dc659a8 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:00:55.386210 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46b4cbc-1add-4b4e-9eba-67f431ecae34 service neutron] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:00:55.400009 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46b4cbc-1add-4b4e-9eba-67f431ecae34 service neutron] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:00:55.475003 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf976800-cac7-4cf7-8681-1fdf120d55de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:00:55.476091 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf976800-cac7-4cf7-8681-1fdf120d55de admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:55.476476 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf976800-cac7-4cf7-8681-1fdf120d55de admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:55.476869 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf976800-cac7-4cf7-8681-1fdf120d55de admin admin] PATH_INFO: `/v3/users/7766b5d8d7fd48b99dd7fcf8b16562b8` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:55.478600 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf976800-cac7-4cf7-8681-1fdf120d55de admin admin] RBAC: Authorizing `identity:delete_user(user_id=7766b5d8d7fd48b99dd7fcf8b16562b8)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:00:55.481455 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf976800-cac7-4cf7-8681-1fdf120d55de admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:00:55.775035 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-bf976800-cac7-4cf7-8681-1fdf120d55de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7766b5d8d7fd48b99dd7fcf8b16562b8'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:55.785483 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-bf976800-cac7-4cf7-8681-1fdf120d55de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7766b5d8d7fd48b99dd7fcf8b16562b8'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:55.785483 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-bf976800-cac7-4cf7-8681-1fdf120d55de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7766b5d8d7fd48b99dd7fcf8b16562b8'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:55.812444 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-bf976800-cac7-4cf7-8681-1fdf120d55de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7766b5d8d7fd48b99dd7fcf8b16562b8'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:55.868988 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-bf976800-cac7-4cf7-8681-1fdf120d55de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7766b5d8d7fd48b99dd7fcf8b16562b8'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:55.876812 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1230/2444] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 23:00:55 2022] DELETE /identity/v3/users/7766b5d8d7fd48b99dd7fcf8b16562b8 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:00:56.605431 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb0cd35-c743-4f00-96d5-eb7f067b3802 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:00:56.607110 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-5eb0cd35-c743-4f00-96d5-eb7f067b3802 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:00:56.656759 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb0cd35-c743-4f00-96d5-eb7f067b3802 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:00:56.780894 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:00:56.805263 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e3197c-c130-4d88-a653-9c39559b559a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:00:56.806149 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3197c-c130-4d88-a653-9c39559b559a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:56.806443 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3197c-c130-4d88-a653-9c39559b559a admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:56.806938 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3197c-c130-4d88-a653-9c39559b559a admin admin] PATH_INFO: `/v3/projects/ed902da7c1c54ef09c1f6914630232ab` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:56.808329 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e3197c-c130-4d88-a653-9c39559b559a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed902da7c1c54ef09c1f6914630232ab)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:00:56.810254 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e3197c-c130-4d88-a653-9c39559b559a admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:00:57.003486 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-22e3197c-c130-4d88-a653-9c39559b559a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed902da7c1c54ef09c1f6914630232ab', 'request_id': 'req-22e3197c-c130-4d88-a653-9c39559b559a'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:57.015829 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-22e3197c-c130-4d88-a653-9c39559b559a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed902da7c1c54ef09c1f6914630232ab', 'request_id': 'req-22e3197c-c130-4d88-a653-9c39559b559a'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:57.097888 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-22e3197c-c130-4d88-a653-9c39559b559a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed902da7c1c54ef09c1f6914630232ab', 'request_id': 'req-22e3197c-c130-4d88-a653-9c39559b559a'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:57.122479 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-22e3197c-c130-4d88-a653-9c39559b559a admin admin] The token cache is being invalidate because project ed902da7c1c54ef09c1f6914630232ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:00:57.122479 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-22e3197c-c130-4d88-a653-9c39559b559a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:57.122996 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-22e3197c-c130-4d88-a653-9c39559b559a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:57.130981 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1215/2445] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 23:00:56 2022] DELETE /identity/v3/projects/ed902da7c1c54ef09c1f6914630232ab => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:00:57.137273 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ac3d2f-24ce-4e9d-8150-2cfe1f77d4f4 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:57.137787 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ac3d2f-24ce-4e9d-8150-2cfe1f77d4f4 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:57.137909 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ac3d2f-24ce-4e9d-8150-2cfe1f77d4f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:57.170762 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-19ac3d2f-24ce-4e9d-8150-2cfe1f77d4f4 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:00:57.178302 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-19ac3d2f-24ce-4e9d-8150-2cfe1f77d4f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:00:57.203501 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:00:57.276109 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1231/2446] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:00:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:00:57.283507 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a138e-1101-4e78-a18d-2f36b1cc1c02 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:57.283855 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a138e-1101-4e78-a18d-2f36b1cc1c02 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:57.284139 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a138e-1101-4e78-a18d-2f36b1cc1c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:57.316777 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-3e6a138e-1101-4e78-a18d-2f36b1cc1c02 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:00:57.317974 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-3e6a138e-1101-4e78-a18d-2f36b1cc1c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:00:57.339642 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1216/2447] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:00:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:00:57.344040 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ac3d2f-24ce-4e9d-8150-2cfe1f77d4f4 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:00:57.356759 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ac3d2f-24ce-4e9d-8150-2cfe1f77d4f4 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:00:57.369208 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b5041f-d76e-40e1-a004-702942a027f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:00:57.370000 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b5041f-d76e-40e1-a004-702942a027f3 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:57.371756 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b5041f-d76e-40e1-a004-702942a027f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:57.372110 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b5041f-d76e-40e1-a004-702942a027f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:57.373034 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b5041f-d76e-40e1-a004-702942a027f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 23:00:57.375085 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b5041f-d76e-40e1-a004-702942a027f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:00:57.376731 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b5041f-d76e-40e1-a004-702942a027f3 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:00:57.394586 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1232/2448] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 23:00:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 23:00:57.399636 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a138e-1101-4e78-a18d-2f36b1cc1c02 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:00:57.411724 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a138e-1101-4e78-a18d-2f36b1cc1c02 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:00:57.423999 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3c0d64-058b-4d1d-8912-4ecda4799084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:00:57.424853 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3c0d64-058b-4d1d-8912-4ecda4799084 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:57.425144 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3c0d64-058b-4d1d-8912-4ecda4799084 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:57.425418 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3c0d64-058b-4d1d-8912-4ecda4799084 admin admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:57.426107 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3c0d64-058b-4d1d-8912-4ecda4799084 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:00:57.428154 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3c0d64-058b-4d1d-8912-4ecda4799084 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:00:57.535160 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1217/2449] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 23:00:57 2022] POST /identity/v3/projects => generated 373 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:00:57.539905 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b5041f-d76e-40e1-a004-702942a027f3 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:00:57.553379 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b5041f-d76e-40e1-a004-702942a027f3 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:00:57.657405 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f070bb-66e9-4428-bfd9-e37706bded0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:00:57.658231 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f070bb-66e9-4428-bfd9-e37706bded0c admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:57.658588 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f070bb-66e9-4428-bfd9-e37706bded0c admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:57.658964 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f070bb-66e9-4428-bfd9-e37706bded0c admin admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:57.659951 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f070bb-66e9-4428-bfd9-e37706bded0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:00:57.662117 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f070bb-66e9-4428-bfd9-e37706bded0c admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:00:57.764835 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1233/2450] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 23:00:57 2022] POST /identity/v3/users => generated 287 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 23:00:57.769233 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3c0d64-058b-4d1d-8912-4ecda4799084 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:00:57.780792 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3c0d64-058b-4d1d-8912-4ecda4799084 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:00:57.785478 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b49db-7a5c-4bd0-918e-8da1ab1effad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:00:57.786338 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b49db-7a5c-4bd0-918e-8da1ab1effad admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:57.786620 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b49db-7a5c-4bd0-918e-8da1ab1effad admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:57.786927 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b49db-7a5c-4bd0-918e-8da1ab1effad admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:57.787687 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789b49db-7a5c-4bd0-918e-8da1ab1effad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:00:57.789720 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789b49db-7a5c-4bd0-918e-8da1ab1effad admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:00:57.803972 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f070bb-66e9-4428-bfd9-e37706bded0c admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:00:57.805632 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1218/2451] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:00:57.806752 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b49db-7a5c-4bd0-918e-8da1ab1effad admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:00:57.812969 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-94f070bb-66e9-4428-bfd9-e37706bded0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:00:57.818174 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b49db-7a5c-4bd0-918e-8da1ab1effad admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:00:57.826547 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32209c87-a974-43cd-980d-405815222ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:00:57.826547 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32209c87-a974-43cd-980d-405815222ce0 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:57.826547 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32209c87-a974-43cd-980d-405815222ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:57.826547 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32209c87-a974-43cd-980d-405815222ce0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:57.827157 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32209c87-a974-43cd-980d-405815222ce0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:00:57.828514 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32209c87-a974-43cd-980d-405815222ce0 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:00:57.830671 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f070bb-66e9-4428-bfd9-e37706bded0c admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:00:57.835705 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f21e20-591a-4549-b39a-0700a5fb9602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:00:57.836768 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f21e20-591a-4549-b39a-0700a5fb9602 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:57.836996 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f21e20-591a-4549-b39a-0700a5fb9602 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:57.837315 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f21e20-591a-4549-b39a-0700a5fb9602 admin admin] PATH_INFO: `/v3/users/a0a765202a9645a1acc332f559537458` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:57.838599 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f21e20-591a-4549-b39a-0700a5fb9602 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0a765202a9645a1acc332f559537458)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:00:57.843295 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1219/2452] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:00:57.843449 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f21e20-591a-4549-b39a-0700a5fb9602 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:00:57.844477 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32209c87-a974-43cd-980d-405815222ce0 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:00:57.856674 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32209c87-a974-43cd-980d-405815222ce0 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:00:57.861469 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b370a7-fbd9-4f8d-9c9f-2e2012fed5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:00:57.862419 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b370a7-fbd9-4f8d-9c9f-2e2012fed5d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:57.862812 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b370a7-fbd9-4f8d-9c9f-2e2012fed5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:57.863144 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b370a7-fbd9-4f8d-9c9f-2e2012fed5d0 admin admin] PATH_INFO: `/v3/projects/6b0309497332486c9f8a05b1cd2d7b6c/users/9460fea3d33d4284badfa04013453cc4/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:57.880548 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b370a7-fbd9-4f8d-9c9f-2e2012fed5d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b0309497332486c9f8a05b1cd2d7b6c, user_id=9460fea3d33d4284badfa04013453cc4, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:00:57.883635 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b370a7-fbd9-4f8d-9c9f-2e2012fed5d0 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:00:57.991206 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1220/2453] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 23:00:57 2022] PUT /identity/v3/projects/6b0309497332486c9f8a05b1cd2d7b6c/users/9460fea3d33d4284badfa04013453cc4/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:00:57.996274 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9597f-d6dc-45ef-bc3f-87f1a7895ebe None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:57.996585 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9597f-d6dc-45ef-bc3f-87f1a7895ebe None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:57.996854 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9597f-d6dc-45ef-bc3f-87f1a7895ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:58.013948 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-a5a9597f-d6dc-45ef-bc3f-87f1a7895ebe None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:00:58.015200 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-a5a9597f-d6dc-45ef-bc3f-87f1a7895ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:00:58.058621 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-f1f21e20-591a-4549-b39a-0700a5fb9602 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0a765202a9645a1acc332f559537458'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:58.063607 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:00:58.066216 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-f1f21e20-591a-4549-b39a-0700a5fb9602 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0a765202a9645a1acc332f559537458'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:58.066871 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-f1f21e20-591a-4549-b39a-0700a5fb9602 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0a765202a9645a1acc332f559537458'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:58.077800 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1221/2454] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:00:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 23:00:58.082396 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-f1f21e20-591a-4549-b39a-0700a5fb9602 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0a765202a9645a1acc332f559537458'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:58.089367 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a9597f-d6dc-45ef-bc3f-87f1a7895ebe None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:00:58.090848 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-a5a9597f-d6dc-45ef-bc3f-87f1a7895ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:00:58.148076 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a9597f-d6dc-45ef-bc3f-87f1a7895ebe None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:00:58.151839 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1598936f-9ff8-49da-90fa-16c864b1e8a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:00:58.152143 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-f1f21e20-591a-4549-b39a-0700a5fb9602 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0a765202a9645a1acc332f559537458'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:58.152585 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598936f-9ff8-49da-90fa-16c864b1e8a2 service neutron] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:58.152765 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598936f-9ff8-49da-90fa-16c864b1e8a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:58.153045 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598936f-9ff8-49da-90fa-16c864b1e8a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:58.154875 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-1598936f-9ff8-49da-90fa-16c864b1e8a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:00:58.159253 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1234/2455] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 23:00:57 2022] DELETE /identity/v3/users/a0a765202a9645a1acc332f559537458 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:00:58.167592 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1598936f-9ff8-49da-90fa-16c864b1e8a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:00:58.169036 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1598936f-9ff8-49da-90fa-16c864b1e8a2 service neutron] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:00:58.179760 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1222/2456] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:00:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 23:00:58.456177 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f21e20-591a-4549-b39a-0700a5fb9602 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:00:58.457625 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-f1f21e20-591a-4549-b39a-0700a5fb9602 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:00:58.469181 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:00:58.530973 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f21e20-591a-4549-b39a-0700a5fb9602 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:00:58.536607 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bb2968-b40d-4a12-8811-1f09efee9e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:00:58.537471 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb2968-b40d-4a12-8811-1f09efee9e96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:58.537757 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb2968-b40d-4a12-8811-1f09efee9e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:58.538039 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb2968-b40d-4a12-8811-1f09efee9e96 admin admin] PATH_INFO: `/v3/projects/93d60e3f3bcd48f292f30a8d2f0351d7` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:58.539310 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bb2968-b40d-4a12-8811-1f09efee9e96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93d60e3f3bcd48f292f30a8d2f0351d7)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:00:58.541250 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bb2968-b40d-4a12-8811-1f09efee9e96 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:00:58.655069 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-c6bb2968-b40d-4a12-8811-1f09efee9e96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93d60e3f3bcd48f292f30a8d2f0351d7', 'request_id': 'req-c6bb2968-b40d-4a12-8811-1f09efee9e96'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:58.660457 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-c6bb2968-b40d-4a12-8811-1f09efee9e96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93d60e3f3bcd48f292f30a8d2f0351d7', 'request_id': 'req-c6bb2968-b40d-4a12-8811-1f09efee9e96'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:58.717988 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-c6bb2968-b40d-4a12-8811-1f09efee9e96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93d60e3f3bcd48f292f30a8d2f0351d7', 'request_id': 'req-c6bb2968-b40d-4a12-8811-1f09efee9e96'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:58.730151 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-c6bb2968-b40d-4a12-8811-1f09efee9e96 admin admin] The token cache is being invalidate because project 93d60e3f3bcd48f292f30a8d2f0351d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:00:58.730437 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-c6bb2968-b40d-4a12-8811-1f09efee9e96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:58.731083 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-c6bb2968-b40d-4a12-8811-1f09efee9e96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:58.732659 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1235/2457] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 23:00:58 2022] DELETE /identity/v3/projects/93d60e3f3bcd48f292f30a8d2f0351d7 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:00:58.736844 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1598936f-9ff8-49da-90fa-16c864b1e8a2 service neutron] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:00:58.747409 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1598936f-9ff8-49da-90fa-16c864b1e8a2 service neutron] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:00:58.814005 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfcc9e3-ac64-42f9-8b98-c868d4a94f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:00:58.815781 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfcc9e3-ac64-42f9-8b98-c868d4a94f62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:58.817916 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfcc9e3-ac64-42f9-8b98-c868d4a94f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:58.817916 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfcc9e3-ac64-42f9-8b98-c868d4a94f62 admin admin] PATH_INFO: `/v3/users/7a26f35b830941839f2e44e34d1902f1` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:58.818384 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfcc9e3-ac64-42f9-8b98-c868d4a94f62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a26f35b830941839f2e44e34d1902f1)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:00:58.820516 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfcc9e3-ac64-42f9-8b98-c868d4a94f62 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:00:59.054487 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-edfcc9e3-ac64-42f9-8b98-c868d4a94f62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a26f35b830941839f2e44e34d1902f1'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:59.061625 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-edfcc9e3-ac64-42f9-8b98-c868d4a94f62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a26f35b830941839f2e44e34d1902f1'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:59.062306 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-edfcc9e3-ac64-42f9-8b98-c868d4a94f62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a26f35b830941839f2e44e34d1902f1'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:59.074460 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-edfcc9e3-ac64-42f9-8b98-c868d4a94f62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a26f35b830941839f2e44e34d1902f1'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:59.075168 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-edfcc9e3-ac64-42f9-8b98-c868d4a94f62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a26f35b830941839f2e44e34d1902f1'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:59.166915 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1223/2458] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 23:00:58 2022] DELETE /identity/v3/users/7a26f35b830941839f2e44e34d1902f1 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:00:59.425676 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bb2968-b40d-4a12-8811-1f09efee9e96 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:00:59.429688 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-c6bb2968-b40d-4a12-8811-1f09efee9e96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:00:59.443458 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bb2968-b40d-4a12-8811-1f09efee9e96 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:00:59.504686 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283a99d1-7065-4a14-b957-937823fe0489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:00:59.505606 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a99d1-7065-4a14-b957-937823fe0489 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:59.505893 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a99d1-7065-4a14-b957-937823fe0489 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:59.506246 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a99d1-7065-4a14-b957-937823fe0489 admin admin] PATH_INFO: `/v3/projects/f06551316b854fcdb193b2ec592a644c` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:59.507455 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283a99d1-7065-4a14-b957-937823fe0489 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f06551316b854fcdb193b2ec592a644c)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:00:59.514820 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283a99d1-7065-4a14-b957-937823fe0489 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:00:59.521667 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c47217-cc2f-4081-9509-e2879475dcde None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:59.521667 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c47217-cc2f-4081-9509-e2879475dcde None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:59.521667 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c47217-cc2f-4081-9509-e2879475dcde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:59.549731 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-13c47217-cc2f-4081-9509-e2879475dcde None None] MFA Rules not processed for user `9460fea3d33d4284badfa04013453cc4`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:00:59.551028 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-13c47217-cc2f-4081-9509-e2879475dcde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:00:59.558679 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:00:59.619017 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1224/2459] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:00:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:00:59.623230 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c47217-cc2f-4081-9509-e2879475dcde None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:00:59.624949 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-13c47217-cc2f-4081-9509-e2879475dcde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:00:59.639357 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c47217-cc2f-4081-9509-e2879475dcde None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:00:59.640550 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:00:59.647934 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-283a99d1-7065-4a14-b957-937823fe0489 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f06551316b854fcdb193b2ec592a644c', 'request_id': 'req-283a99d1-7065-4a14-b957-937823fe0489'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:59.653360 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-283a99d1-7065-4a14-b957-937823fe0489 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f06551316b854fcdb193b2ec592a644c', 'request_id': 'req-283a99d1-7065-4a14-b957-937823fe0489'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:59.697466 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdd2d24-c331-4756-b45c-6313f7dd28e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:00:59.700677 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdd2d24-c331-4756-b45c-6313f7dd28e3 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:59.700677 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdd2d24-c331-4756-b45c-6313f7dd28e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:59.700677 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdd2d24-c331-4756-b45c-6313f7dd28e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:59.700677 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdd2d24-c331-4756-b45c-6313f7dd28e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:00:59.701483 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdd2d24-c331-4756-b45c-6313f7dd28e3 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:00:59.719968 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-283a99d1-7065-4a14-b957-937823fe0489 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f06551316b854fcdb193b2ec592a644c', 'request_id': 'req-283a99d1-7065-4a14-b957-937823fe0489'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:59.731958 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-283a99d1-7065-4a14-b957-937823fe0489 admin admin] The token cache is being invalidate because project f06551316b854fcdb193b2ec592a644c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:00:59.732274 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-283a99d1-7065-4a14-b957-937823fe0489 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:59.732921 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-283a99d1-7065-4a14-b957-937823fe0489 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:00:59.734516 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1236/2460] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 23:00:59 2022] DELETE /identity/v3/projects/f06551316b854fcdb193b2ec592a644c => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:00:59.739300 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172d4c79-8778-4af6-aff6-683171f37438 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:59.739598 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172d4c79-8778-4af6-aff6-683171f37438 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:59.739944 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172d4c79-8778-4af6-aff6-683171f37438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:59.757652 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-172d4c79-8778-4af6-aff6-683171f37438 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:00:59.758911 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-172d4c79-8778-4af6-aff6-683171f37438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:00:59.766884 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1237/2461] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:00:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 23:00:59.775433 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b636cf56-a893-433e-9950-a246879c442a None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:59.779056 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b636cf56-a893-433e-9950-a246879c442a None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:59.779056 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b636cf56-a893-433e-9950-a246879c442a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:59.793215 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-b636cf56-a893-433e-9950-a246879c442a None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:00:59.794403 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-b636cf56-a893-433e-9950-a246879c442a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:00:59.802323 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1238/2462] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:00:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 23:00:59.805297 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1225/2463] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 23:00:59 2022] POST /identity/v3/projects => generated 373 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 23:00:59.806616 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b636cf56-a893-433e-9950-a246879c442a None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:00:59.808904 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdd2d24-c331-4756-b45c-6313f7dd28e3 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:00:59.810513 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-2bdd2d24-c331-4756-b45c-6313f7dd28e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:00:59.815724 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b636cf56-a893-433e-9950-a246879c442a None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:00:59.871423 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de2b6e-4673-43ea-a85b-b4bfa6c3c87f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:00:59.872086 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de2b6e-4673-43ea-a85b-b4bfa6c3c87f admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:59.872369 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de2b6e-4673-43ea-a85b-b4bfa6c3c87f admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:59.872646 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de2b6e-4673-43ea-a85b-b4bfa6c3c87f admin admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:59.873377 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6de2b6e-4673-43ea-a85b-b4bfa6c3c87f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 23:00:59.873676 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6de2b6e-4673-43ea-a85b-b4bfa6c3c87f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:00:59.875294 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6de2b6e-4673-43ea-a85b-b4bfa6c3c87f admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:00:59.876987 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdd2d24-c331-4756-b45c-6313f7dd28e3 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:00:59.882304 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ae2526-2f0c-46db-9705-dec5bcf9549a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:00:59.882914 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae2526-2f0c-46db-9705-dec5bcf9549a admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:59.883202 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae2526-2f0c-46db-9705-dec5bcf9549a admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:59.883481 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae2526-2f0c-46db-9705-dec5bcf9549a admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:59.884608 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ae2526-2f0c-46db-9705-dec5bcf9549a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:00:59.886064 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ae2526-2f0c-46db-9705-dec5bcf9549a admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:00:59.887474 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1239/2464] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 23:00:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 23:00:59.891866 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de2b6e-4673-43ea-a85b-b4bfa6c3c87f admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:00:59.902001 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de2b6e-4673-43ea-a85b-b4bfa6c3c87f admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:00:59.906654 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ebe2d3-046c-4cff-a666-0f5056f88349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:00:59.907331 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ebe2d3-046c-4cff-a666-0f5056f88349 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:00:59.911020 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ebe2d3-046c-4cff-a666-0f5056f88349 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:00:59.911020 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ebe2d3-046c-4cff-a666-0f5056f88349 admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:00:59.911020 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ebe2d3-046c-4cff-a666-0f5056f88349 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:00:59.911020 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ebe2d3-046c-4cff-a666-0f5056f88349 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:00.003985 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1226/2465] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 23:00:59 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 23:01:00.008319 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ae2526-2f0c-46db-9705-dec5bcf9549a admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:00.019084 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1240/2466] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 23:00:59 2022] POST /identity/v3/projects => generated 377 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:01:00.019288 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ae2526-2f0c-46db-9705-dec5bcf9549a admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:00.020151 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ebe2d3-046c-4cff-a666-0f5056f88349 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:00.075281 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef121037-9399-40ca-90e9-217de8a97448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:00.075949 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef121037-9399-40ca-90e9-217de8a97448 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:00.076229 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef121037-9399-40ca-90e9-217de8a97448 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:00.076505 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef121037-9399-40ca-90e9-217de8a97448 admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:00.077403 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef121037-9399-40ca-90e9-217de8a97448 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:00.079039 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef121037-9399-40ca-90e9-217de8a97448 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:00.080773 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ebe2d3-046c-4cff-a666-0f5056f88349 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:00.085530 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f78138-d80c-424a-b6db-0c52ae9e1842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:00.086200 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f78138-d80c-424a-b6db-0c52ae9e1842 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:00.086481 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f78138-d80c-424a-b6db-0c52ae9e1842 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:00.086762 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f78138-d80c-424a-b6db-0c52ae9e1842 admin admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:00.087492 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f78138-d80c-424a-b6db-0c52ae9e1842 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:00.091857 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1227/2467] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 23:01:00.092023 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f78138-d80c-424a-b6db-0c52ae9e1842 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:00.092813 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef121037-9399-40ca-90e9-217de8a97448 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:00.102729 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef121037-9399-40ca-90e9-217de8a97448 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:00.107520 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e6fda3-c327-4236-8ec6-dcbadbcc72c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:00.111738 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e6fda3-c327-4236-8ec6-dcbadbcc72c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:00.111738 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e6fda3-c327-4236-8ec6-dcbadbcc72c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:00.111738 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e6fda3-c327-4236-8ec6-dcbadbcc72c2 admin admin] PATH_INFO: `/v3/projects/daffd8c5ba634747be2acba7b72f5b91/users/8435e6ceb27844c7a8503291d7d3ad6a/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:00.124026 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e6fda3-c327-4236-8ec6-dcbadbcc72c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=daffd8c5ba634747be2acba7b72f5b91, user_id=8435e6ceb27844c7a8503291d7d3ad6a, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:00.126695 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e6fda3-c327-4236-8ec6-dcbadbcc72c2 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:00.171990 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1241/2468] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 23:01:00 2022] POST /identity/v3/users => generated 289 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 23:01:00.176067 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f78138-d80c-424a-b6db-0c52ae9e1842 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:00.185865 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f78138-d80c-424a-b6db-0c52ae9e1842 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:00.190525 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caffa63-a3fc-4a59-95c8-9e1ec57771ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:00.191188 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caffa63-a3fc-4a59-95c8-9e1ec57771ab admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:00.191475 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caffa63-a3fc-4a59-95c8-9e1ec57771ab admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:00.191752 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caffa63-a3fc-4a59-95c8-9e1ec57771ab admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:00.192470 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caffa63-a3fc-4a59-95c8-9e1ec57771ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:00.194674 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caffa63-a3fc-4a59-95c8-9e1ec57771ab admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:00.200035 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:01:00.205843 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1242/2469] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 23:01:00.206021 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1228/2470] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 23:01:00 2022] PUT /identity/v3/projects/daffd8c5ba634747be2acba7b72f5b91/users/8435e6ceb27844c7a8503291d7d3ad6a/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:01:00.209562 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e6fda3-c327-4236-8ec6-dcbadbcc72c2 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:00.209689 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caffa63-a3fc-4a59-95c8-9e1ec57771ab admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:00.219359 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caffa63-a3fc-4a59-95c8-9e1ec57771ab admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:00.272787 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e6fda3-c327-4236-8ec6-dcbadbcc72c2 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:00.277498 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf4866d-ff79-4ee2-8de0-b05d2b638d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:00.278096 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeaaf04-e655-43e7-87ff-1a66ab59bb76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:00.278247 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4866d-ff79-4ee2-8de0-b05d2b638d6a admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:00.278472 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4866d-ff79-4ee2-8de0-b05d2b638d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:00.278754 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeaaf04-e655-43e7-87ff-1a66ab59bb76 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:00.278902 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4866d-ff79-4ee2-8de0-b05d2b638d6a admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:00.279065 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeaaf04-e655-43e7-87ff-1a66ab59bb76 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:00.279339 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeaaf04-e655-43e7-87ff-1a66ab59bb76 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:00.279510 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf4866d-ff79-4ee2-8de0-b05d2b638d6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:00.280055 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeaaf04-e655-43e7-87ff-1a66ab59bb76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:00.281112 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf4866d-ff79-4ee2-8de0-b05d2b638d6a admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:00.281611 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeaaf04-e655-43e7-87ff-1a66ab59bb76 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:00.285020 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:01:00.291877 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1229/2471] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 23:01:00.293351 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1243/2472] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 23:01:00.296295 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf4866d-ff79-4ee2-8de0-b05d2b638d6a admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:00.297236 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeaaf04-e655-43e7-87ff-1a66ab59bb76 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:00.307027 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf4866d-ff79-4ee2-8de0-b05d2b638d6a admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:00.307284 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeaaf04-e655-43e7-87ff-1a66ab59bb76 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:00.311933 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff96c7f-d388-4cf3-bbfd-90f2400ded41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:00.312143 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db8cdb9-c912-42ca-81e0-b4f7f1b4ae83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:00.313049 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8cdb9-c912-42ca-81e0-b4f7f1b4ae83 admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:00.313337 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8cdb9-c912-42ca-81e0-b4f7f1b4ae83 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:00.313540 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff96c7f-d388-4cf3-bbfd-90f2400ded41 admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:00.313683 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8cdb9-c912-42ca-81e0-b4f7f1b4ae83 admin admin] PATH_INFO: `/v3/projects/daffd8c5ba634747be2acba7b72f5b91/users/8435e6ceb27844c7a8503291d7d3ad6a/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:00.314348 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff96c7f-d388-4cf3-bbfd-90f2400ded41 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:00.314820 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff96c7f-d388-4cf3-bbfd-90f2400ded41 admin admin] PATH_INFO: `/v3/projects/24a2b49707184c4093e713f0bbc5b00c/users/ae2a190cf2ef4777a3838cbe9750afea/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:00.315633 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db8cdb9-c912-42ca-81e0-b4f7f1b4ae83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=daffd8c5ba634747be2acba7b72f5b91, user_id=8435e6ceb27844c7a8503291d7d3ad6a, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:00.318925 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db8cdb9-c912-42ca-81e0-b4f7f1b4ae83 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:00.329156 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1244/2473] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 23:01:00 2022] PUT /identity/v3/projects/daffd8c5ba634747be2acba7b72f5b91/users/8435e6ceb27844c7a8503291d7d3ad6a/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 23:01:00.331178 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff96c7f-d388-4cf3-bbfd-90f2400ded41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24a2b49707184c4093e713f0bbc5b00c, user_id=ae2a190cf2ef4777a3838cbe9750afea, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:00.334018 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff96c7f-d388-4cf3-bbfd-90f2400ded41 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:00.404221 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1230/2474] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 23:01:00 2022] PUT /identity/v3/projects/24a2b49707184c4093e713f0bbc5b00c/users/ae2a190cf2ef4777a3838cbe9750afea/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 23:01:00.409717 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64bb514-60a6-4c0f-b6c5-7ce723fc6342 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:00.410124 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64bb514-60a6-4c0f-b6c5-7ce723fc6342 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:00.410410 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64bb514-60a6-4c0f-b6c5-7ce723fc6342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:00.431368 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-f64bb514-60a6-4c0f-b6c5-7ce723fc6342 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:01:00.431913 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-f64bb514-60a6-4c0f-b6c5-7ce723fc6342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:00.497834 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1245/2475] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:01:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:01:00.508186 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff96c7f-d388-4cf3-bbfd-90f2400ded41 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:00.510566 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-5ff96c7f-d388-4cf3-bbfd-90f2400ded41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:00.575387 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff96c7f-d388-4cf3-bbfd-90f2400ded41 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:00.579037 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b1818a-80e5-4cce-9b72-4ceca44d3a61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:00.579668 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1818a-80e5-4cce-9b72-4ceca44d3a61 service neutron] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:00.579951 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1818a-80e5-4cce-9b72-4ceca44d3a61 service neutron] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:00.580230 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1818a-80e5-4cce-9b72-4ceca44d3a61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:00.589492 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b1818a-80e5-4cce-9b72-4ceca44d3a61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:00.590936 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b1818a-80e5-4cce-9b72-4ceca44d3a61 service neutron] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:00.601176 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1231/2476] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:01:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:01:01.588122 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a0409-4819-43f2-9c56-ab3b2e3f7b48 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:01.588633 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a0409-4819-43f2-9c56-ab3b2e3f7b48 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:01.588741 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a0409-4819-43f2-9c56-ab3b2e3f7b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:01.612105 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-382a0409-4819-43f2-9c56-ab3b2e3f7b48 None None] MFA Rules not processed for user `63cc904578554aea84ff1917a5ed075e`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:01:01.613515 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-382a0409-4819-43f2-9c56-ab3b2e3f7b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:01.673916 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1246/2477] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Jul 21 23:01:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:01:01.702062 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b1818a-80e5-4cce-9b72-4ceca44d3a61 service neutron] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:01.703950 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f86a307-6419-46b6-81c2-88ad6f2d2fe4 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:01.704456 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f86a307-6419-46b6-81c2-88ad6f2d2fe4 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:01.704570 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-23b1818a-80e5-4cce-9b72-4ceca44d3a61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:01.704783 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f86a307-6419-46b6-81c2-88ad6f2d2fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:01.721549 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:01:01.736374 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-8f86a307-6419-46b6-81c2-88ad6f2d2fe4 None None] MFA Rules not processed for user `8435e6ceb27844c7a8503291d7d3ad6a`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:01:01.737426 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-8f86a307-6419-46b6-81c2-88ad6f2d2fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:01.759545 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:01:01.794210 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b1818a-80e5-4cce-9b72-4ceca44d3a61 service neutron] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:01.800141 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cbde4a-eba1-4a84-8a56-7f170536c68d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba7eef4e1b27476ab4016449d9c3a4b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:01.800987 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cbde4a-eba1-4a84-8a56-7f170536c68d service swift] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:01.801280 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cbde4a-eba1-4a84-8a56-7f170536c68d service swift] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:01.801581 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cbde4a-eba1-4a84-8a56-7f170536c68d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:01.808726 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1247/2478] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:01:01 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:01:01.817807 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cbde4a-eba1-4a84-8a56-7f170536c68d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:01.820156 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cbde4a-eba1-4a84-8a56-7f170536c68d service swift] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:01.823844 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f86a307-6419-46b6-81c2-88ad6f2d2fe4 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:01.825274 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-8f86a307-6419-46b6-81c2-88ad6f2d2fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:01.829372 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1232/2479] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jul 21 23:01:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 21 23:01:01.902996 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f86a307-6419-46b6-81c2-88ad6f2d2fe4 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:01.905160 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e761c-c0b9-4067-a5f8-d387855cfb7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:01.905860 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e761c-c0b9-4067-a5f8-d387855cfb7a service cinder] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:01.906144 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e761c-c0b9-4067-a5f8-d387855cfb7a service cinder] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:01.906416 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e761c-c0b9-4067-a5f8-d387855cfb7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:01.908584 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-583e761c-c0b9-4067-a5f8-d387855cfb7a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:02.019413 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583e761c-c0b9-4067-a5f8-d387855cfb7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:02.021055 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583e761c-c0b9-4067-a5f8-d387855cfb7a service cinder] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:02.033688 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1248/2480] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:01:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 23:01:04.029118 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cbde4a-eba1-4a84-8a56-7f170536c68d service swift] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:04.031248 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-a6cbde4a-eba1-4a84-8a56-7f170536c68d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:04.048472 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cbde4a-eba1-4a84-8a56-7f170536c68d service swift] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:04.054153 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e6f1cd-1b7a-4a67-869f-37ba4d402be4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:04.055577 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e6f1cd-1b7a-4a67-869f-37ba4d402be4 service cinder] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:04.055871 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e6f1cd-1b7a-4a67-869f-37ba4d402be4 service cinder] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:04.058162 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e6f1cd-1b7a-4a67-869f-37ba4d402be4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:04.080078 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e6f1cd-1b7a-4a67-869f-37ba4d402be4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:04.082171 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e6f1cd-1b7a-4a67-869f-37ba4d402be4 service cinder] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:04.098478 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1233/2481] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:01:04 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:01:07.312730 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc43443-5f45-4b5d-9d10-5a1ed70801e7 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:07.313616 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc43443-5f45-4b5d-9d10-5a1ed70801e7 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:07.314055 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc43443-5f45-4b5d-9d10-5a1ed70801e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:07.366873 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-2fc43443-5f45-4b5d-9d10-5a1ed70801e7 None None] MFA Rules not processed for user `ae2a190cf2ef4777a3838cbe9750afea`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:01:07.369059 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-2fc43443-5f45-4b5d-9d10-5a1ed70801e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:07.477535 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1249/2482] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:01:07 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:01:07.485161 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e6f1cd-1b7a-4a67-869f-37ba4d402be4 service cinder] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:07.500934 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e6f1cd-1b7a-4a67-869f-37ba4d402be4 service cinder] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:07.513402 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac8d3dc-9432-4c9f-a02c-92099a38b768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:07.514766 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac8d3dc-9432-4c9f-a02c-92099a38b768 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:07.515344 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac8d3dc-9432-4c9f-a02c-92099a38b768 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:07.515877 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac8d3dc-9432-4c9f-a02c-92099a38b768 admin admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:07.516589 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac8d3dc-9432-4c9f-a02c-92099a38b768 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:07.523085 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac8d3dc-9432-4c9f-a02c-92099a38b768 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:07.631816 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1234/2483] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 23:01:07 2022] POST /identity/v3/projects => generated 377 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:01:07.633661 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc43443-5f45-4b5d-9d10-5a1ed70801e7 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:07.643474 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:01:07.653518 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc43443-5f45-4b5d-9d10-5a1ed70801e7 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:07.763440 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11110609-e9e0-4f71-91f1-cbf611ff5905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:07.764245 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11110609-e9e0-4f71-91f1-cbf611ff5905 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:07.764595 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11110609-e9e0-4f71-91f1-cbf611ff5905 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:07.764962 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11110609-e9e0-4f71-91f1-cbf611ff5905 admin admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:07.765738 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11110609-e9e0-4f71-91f1-cbf611ff5905 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:07.767797 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11110609-e9e0-4f71-91f1-cbf611ff5905 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:07.861692 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1250/2484] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 23:01:07 2022] POST /identity/v3/users => generated 295 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:01:07.866211 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac8d3dc-9432-4c9f-a02c-92099a38b768 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:07.877778 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac8d3dc-9432-4c9f-a02c-92099a38b768 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:07.884264 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568bd7a8-4ebb-4867-8a0e-2e1a5b72fc54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:07.884959 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568bd7a8-4ebb-4867-8a0e-2e1a5b72fc54 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:07.885261 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568bd7a8-4ebb-4867-8a0e-2e1a5b72fc54 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:07.885779 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568bd7a8-4ebb-4867-8a0e-2e1a5b72fc54 admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:07.886528 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568bd7a8-4ebb-4867-8a0e-2e1a5b72fc54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:07.890043 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568bd7a8-4ebb-4867-8a0e-2e1a5b72fc54 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:07.917074 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1235/2485] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:01:07 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:01:07.917710 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11110609-e9e0-4f71-91f1-cbf611ff5905 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:07.932747 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11110609-e9e0-4f71-91f1-cbf611ff5905 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:07.943966 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e45cfea-0cf1-4bd0-a9ff-84532a63862e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:07.945496 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45cfea-0cf1-4bd0-a9ff-84532a63862e admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:07.945986 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45cfea-0cf1-4bd0-a9ff-84532a63862e admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:07.946673 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45cfea-0cf1-4bd0-a9ff-84532a63862e admin admin] PATH_INFO: `/v3/projects/42ade7e26dde4d428b1319ef8e45626a/users/770c05b9ae99483e977aa3218a088ac1/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:07.966596 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e45cfea-0cf1-4bd0-a9ff-84532a63862e admin admin] RBAC: Authorizing `identity:create_grant(project_id=42ade7e26dde4d428b1319ef8e45626a, user_id=770c05b9ae99483e977aa3218a088ac1, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:07.969405 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e45cfea-0cf1-4bd0-a9ff-84532a63862e admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:08.084725 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568bd7a8-4ebb-4867-8a0e-2e1a5b72fc54 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:08.085521 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1251/2486] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 23:01:07 2022] PUT /identity/v3/projects/42ade7e26dde4d428b1319ef8e45626a/users/770c05b9ae99483e977aa3218a088ac1/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:01:08.113872 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568bd7a8-4ebb-4867-8a0e-2e1a5b72fc54 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:08.223354 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40841aae-e3ab-41fa-97fe-be99a32539bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:08.224644 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40841aae-e3ab-41fa-97fe-be99a32539bf admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:08.224972 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40841aae-e3ab-41fa-97fe-be99a32539bf admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:08.225198 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40841aae-e3ab-41fa-97fe-be99a32539bf admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:08.226011 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40841aae-e3ab-41fa-97fe-be99a32539bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:08.227867 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40841aae-e3ab-41fa-97fe-be99a32539bf admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:08.239642 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1236/2487] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:01:08 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:01:08.246019 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e45cfea-0cf1-4bd0-a9ff-84532a63862e admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:08.262455 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e45cfea-0cf1-4bd0-a9ff-84532a63862e admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:08.269347 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0976f1c0-96ea-4292-9316-4e57ff62ddf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:08.270294 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0976f1c0-96ea-4292-9316-4e57ff62ddf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:08.270594 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0976f1c0-96ea-4292-9316-4e57ff62ddf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:08.272522 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0976f1c0-96ea-4292-9316-4e57ff62ddf8 admin admin] PATH_INFO: `/v3/projects/42ade7e26dde4d428b1319ef8e45626a/users/770c05b9ae99483e977aa3218a088ac1/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:08.273993 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0976f1c0-96ea-4292-9316-4e57ff62ddf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42ade7e26dde4d428b1319ef8e45626a, user_id=770c05b9ae99483e977aa3218a088ac1, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:08.276839 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0976f1c0-96ea-4292-9316-4e57ff62ddf8 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:08.296600 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1252/2488] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 23:01:08 2022] PUT /identity/v3/projects/42ade7e26dde4d428b1319ef8e45626a/users/770c05b9ae99483e977aa3218a088ac1/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:01:16.938355 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64aafd-81c1-433e-a25b-a6372a285da3 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:16.942608 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64aafd-81c1-433e-a25b-a6372a285da3 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:16.942608 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64aafd-81c1-433e-a25b-a6372a285da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:17.005007 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-2d64aafd-81c1-433e-a25b-a6372a285da3 None None] MFA Rules not processed for user `770c05b9ae99483e977aa3218a088ac1`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:01:17.006381 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-2d64aafd-81c1-433e-a25b-a6372a285da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:17.061097 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:01:17.203088 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1237/2489] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:01:16 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 23:01:17.213264 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0976f1c0-96ea-4292-9316-4e57ff62ddf8 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:17.215305 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-0976f1c0-96ea-4292-9316-4e57ff62ddf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:17.424775 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:01:17.506347 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0976f1c0-96ea-4292-9316-4e57ff62ddf8 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:17.514929 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd33337-ad7e-4b16-8731-b8128c340509 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bfdb655a2a44b33ab724daaf73a294b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:17.515660 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd33337-ad7e-4b16-8731-b8128c340509 service nova] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:17.515882 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd33337-ad7e-4b16-8731-b8128c340509 service nova] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:17.516160 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd33337-ad7e-4b16-8731-b8128c340509 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:17.629190 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd33337-ad7e-4b16-8731-b8128c340509 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:17.631559 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd33337-ad7e-4b16-8731-b8128c340509 service nova] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:17.662117 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1253/2490] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:01:17 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:01:20.248807 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d64aafd-81c1-433e-a25b-a6372a285da3 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:20.249559 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-2d64aafd-81c1-433e-a25b-a6372a285da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:20.366599 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d64aafd-81c1-433e-a25b-a6372a285da3 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:20.379972 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0195195-7a44-4d6e-90da-677c51507f87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:20.386049 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0195195-7a44-4d6e-90da-677c51507f87 service neutron] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:20.386049 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0195195-7a44-4d6e-90da-677c51507f87 service neutron] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:20.386049 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0195195-7a44-4d6e-90da-677c51507f87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:20.387594 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-f0195195-7a44-4d6e-90da-677c51507f87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:20.533604 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0195195-7a44-4d6e-90da-677c51507f87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:20.536434 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0195195-7a44-4d6e-90da-677c51507f87 service neutron] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:20.563417 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1238/2491] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:01:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:01:23.109194 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd33337-ad7e-4b16-8731-b8128c340509 service nova] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:23.115834 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-7dd33337-ad7e-4b16-8731-b8128c340509 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:23.334635 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd33337-ad7e-4b16-8731-b8128c340509 service nova] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:23.349659 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fce270-b389-4b92-a403-23aaacb7ce7f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10c0bed169344a77bfad29624c5e5a8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:23.350339 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fce270-b389-4b92-a403-23aaacb7ce7f service placement] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:23.350627 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fce270-b389-4b92-a403-23aaacb7ce7f service placement] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:23.350926 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fce270-b389-4b92-a403-23aaacb7ce7f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:23.357739 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-d4fce270-b389-4b92-a403-23aaacb7ce7f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:23.386909 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fce270-b389-4b92-a403-23aaacb7ce7f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:23.388698 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fce270-b389-4b92-a403-23aaacb7ce7f service placement] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:23.411277 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1254/2492] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:01:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:01:24.008722 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0195195-7a44-4d6e-90da-677c51507f87 service neutron] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:24.022303 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0195195-7a44-4d6e-90da-677c51507f87 service neutron] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:24.027529 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a5858-f2d2-4e54-becb-c6a4feb93a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:24.028409 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a5858-f2d2-4e54-becb-c6a4feb93a78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:24.028690 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a5858-f2d2-4e54-becb-c6a4feb93a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:24.028963 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a5858-f2d2-4e54-becb-c6a4feb93a78 admin admin] PATH_INFO: `/v3/users/9460fea3d33d4284badfa04013453cc4` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:24.030823 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515a5858-f2d2-4e54-becb-c6a4feb93a78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9460fea3d33d4284badfa04013453cc4)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:24.032747 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515a5858-f2d2-4e54-becb-c6a4feb93a78 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:24.071410 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:01:24.304359 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-515a5858-f2d2-4e54-becb-c6a4feb93a78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9460fea3d33d4284badfa04013453cc4'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:24.325116 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-515a5858-f2d2-4e54-becb-c6a4feb93a78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9460fea3d33d4284badfa04013453cc4'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:24.325973 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-515a5858-f2d2-4e54-becb-c6a4feb93a78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9460fea3d33d4284badfa04013453cc4'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:24.355826 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-515a5858-f2d2-4e54-becb-c6a4feb93a78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9460fea3d33d4284badfa04013453cc4'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:24.357455 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-515a5858-f2d2-4e54-becb-c6a4feb93a78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9460fea3d33d4284badfa04013453cc4'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:24.449633 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1239/2493] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 23:01:24 2022] DELETE /identity/v3/users/9460fea3d33d4284badfa04013453cc4 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 23:01:24.988420 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fce270-b389-4b92-a403-23aaacb7ce7f service placement] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:25.002175 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fce270-b389-4b92-a403-23aaacb7ce7f service placement] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:25.079472 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:01:25.087683 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd0ac19-cafb-4e49-a31c-cc8d1ac71f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:25.088557 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0ac19-cafb-4e49-a31c-cc8d1ac71f2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:25.088841 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0ac19-cafb-4e49-a31c-cc8d1ac71f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:25.090121 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0ac19-cafb-4e49-a31c-cc8d1ac71f2e admin admin] PATH_INFO: `/v3/projects/6b0309497332486c9f8a05b1cd2d7b6c` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:25.090332 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd0ac19-cafb-4e49-a31c-cc8d1ac71f2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b0309497332486c9f8a05b1cd2d7b6c)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:25.092479 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd0ac19-cafb-4e49-a31c-cc8d1ac71f2e admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:25.222062 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-1cd0ac19-cafb-4e49-a31c-cc8d1ac71f2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b0309497332486c9f8a05b1cd2d7b6c', 'request_id': 'req-1cd0ac19-cafb-4e49-a31c-cc8d1ac71f2e'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:25.231079 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-1cd0ac19-cafb-4e49-a31c-cc8d1ac71f2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b0309497332486c9f8a05b1cd2d7b6c', 'request_id': 'req-1cd0ac19-cafb-4e49-a31c-cc8d1ac71f2e'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:25.312801 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-1cd0ac19-cafb-4e49-a31c-cc8d1ac71f2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b0309497332486c9f8a05b1cd2d7b6c', 'request_id': 'req-1cd0ac19-cafb-4e49-a31c-cc8d1ac71f2e'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:25.328168 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-1cd0ac19-cafb-4e49-a31c-cc8d1ac71f2e admin admin] The token cache is being invalidate because project 6b0309497332486c9f8a05b1cd2d7b6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:01:25.328597 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-1cd0ac19-cafb-4e49-a31c-cc8d1ac71f2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:25.329695 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-1cd0ac19-cafb-4e49-a31c-cc8d1ac71f2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:25.334825 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1255/2494] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 23:01:24 2022] DELETE /identity/v3/projects/6b0309497332486c9f8a05b1cd2d7b6c => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:01:25.337890 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a5858-f2d2-4e54-becb-c6a4feb93a78 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:25.342532 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-515a5858-f2d2-4e54-becb-c6a4feb93a78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:25.374562 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a5858-f2d2-4e54-becb-c6a4feb93a78 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:25.470582 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574122cd-ef2e-4416-9885-77c186ce442c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:25.471799 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574122cd-ef2e-4416-9885-77c186ce442c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:25.475044 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574122cd-ef2e-4416-9885-77c186ce442c admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:25.475402 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574122cd-ef2e-4416-9885-77c186ce442c admin admin] PATH_INFO: `/v3/users/8435e6ceb27844c7a8503291d7d3ad6a` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:25.483967 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574122cd-ef2e-4416-9885-77c186ce442c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8435e6ceb27844c7a8503291d7d3ad6a)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:25.495060 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574122cd-ef2e-4416-9885-77c186ce442c admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:25.870594 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-574122cd-ef2e-4416-9885-77c186ce442c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8435e6ceb27844c7a8503291d7d3ad6a'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:25.878002 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-574122cd-ef2e-4416-9885-77c186ce442c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8435e6ceb27844c7a8503291d7d3ad6a'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:25.878897 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-574122cd-ef2e-4416-9885-77c186ce442c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8435e6ceb27844c7a8503291d7d3ad6a'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:25.891979 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-574122cd-ef2e-4416-9885-77c186ce442c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8435e6ceb27844c7a8503291d7d3ad6a'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:25.892662 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-574122cd-ef2e-4416-9885-77c186ce442c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8435e6ceb27844c7a8503291d7d3ad6a'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:26.013952 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1240/2495] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 23:01:25 2022] DELETE /identity/v3/users/8435e6ceb27844c7a8503291d7d3ad6a => generated 0 bytes in 674 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 23:01:26.522376 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd0ac19-cafb-4e49-a31c-cc8d1ac71f2e admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:26.524562 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-1cd0ac19-cafb-4e49-a31c-cc8d1ac71f2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:26.564851 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd0ac19-cafb-4e49-a31c-cc8d1ac71f2e admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:26.675123 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529635a0-20d6-4439-900a-df9eed65cd95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:26.676179 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529635a0-20d6-4439-900a-df9eed65cd95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:26.677567 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529635a0-20d6-4439-900a-df9eed65cd95 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:26.678001 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529635a0-20d6-4439-900a-df9eed65cd95 admin admin] PATH_INFO: `/v3/projects/daffd8c5ba634747be2acba7b72f5b91` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:26.681696 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529635a0-20d6-4439-900a-df9eed65cd95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=daffd8c5ba634747be2acba7b72f5b91)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:26.681696 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529635a0-20d6-4439-900a-df9eed65cd95 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:26.865834 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-529635a0-20d6-4439-900a-df9eed65cd95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'daffd8c5ba634747be2acba7b72f5b91', 'request_id': 'req-529635a0-20d6-4439-900a-df9eed65cd95'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:26.879084 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-529635a0-20d6-4439-900a-df9eed65cd95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'daffd8c5ba634747be2acba7b72f5b91', 'request_id': 'req-529635a0-20d6-4439-900a-df9eed65cd95'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:27.049070 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-529635a0-20d6-4439-900a-df9eed65cd95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'daffd8c5ba634747be2acba7b72f5b91', 'request_id': 'req-529635a0-20d6-4439-900a-df9eed65cd95'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:27.061083 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-529635a0-20d6-4439-900a-df9eed65cd95 admin admin] The token cache is being invalidate because project daffd8c5ba634747be2acba7b72f5b91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:01:27.061410 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-529635a0-20d6-4439-900a-df9eed65cd95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:27.062493 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-529635a0-20d6-4439-900a-df9eed65cd95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:27.064222 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1256/2496] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 23:01:26 2022] DELETE /identity/v3/projects/daffd8c5ba634747be2acba7b72f5b91 => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:01:27.069167 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4377497-ad06-4baa-85bf-37460f4ebd10 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:27.069599 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4377497-ad06-4baa-85bf-37460f4ebd10 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:27.069797 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4377497-ad06-4baa-85bf-37460f4ebd10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:27.094657 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529635a0-20d6-4439-900a-df9eed65cd95 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:27.096765 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-529635a0-20d6-4439-900a-df9eed65cd95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:27.106431 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:01:27.116881 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:01:27.155544 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-a4377497-ad06-4baa-85bf-37460f4ebd10 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:01:27.156669 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-a4377497-ad06-4baa-85bf-37460f4ebd10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:27.165652 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1241/2497] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:01:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:01:27.168393 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529635a0-20d6-4439-900a-df9eed65cd95 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:27.173329 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662b3756-245f-4c6b-ad59-eca4db985c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:27.173695 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8837a94-43c1-4840-a0c7-37769b314606 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:27.173988 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8837a94-43c1-4840-a0c7-37769b314606 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:27.174270 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8837a94-43c1-4840-a0c7-37769b314606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:27.174532 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b3756-245f-4c6b-ad59-eca4db985c1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:27.174913 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b3756-245f-4c6b-ad59-eca4db985c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:27.175256 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b3756-245f-4c6b-ad59-eca4db985c1b admin admin] PATH_INFO: `/v3/users/d10c68b7fd4446feb759832913fea3b3` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:27.176649 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662b3756-245f-4c6b-ad59-eca4db985c1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d10c68b7fd4446feb759832913fea3b3)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:27.179040 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662b3756-245f-4c6b-ad59-eca4db985c1b admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:27.201630 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-c8837a94-43c1-4840-a0c7-37769b314606 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:01:27.202894 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-c8837a94-43c1-4840-a0c7-37769b314606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:27.215276 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1242/2498] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:01:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 23:01:27.217151 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8837a94-43c1-4840-a0c7-37769b314606 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:27.228535 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8837a94-43c1-4840-a0c7-37769b314606 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:27.234352 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47ccca-0a4c-44cb-8fca-ffad5b1f08d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:27.235049 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47ccca-0a4c-44cb-8fca-ffad5b1f08d6 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:27.235335 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47ccca-0a4c-44cb-8fca-ffad5b1f08d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:27.235610 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47ccca-0a4c-44cb-8fca-ffad5b1f08d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:27.236382 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a47ccca-0a4c-44cb-8fca-ffad5b1f08d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 23:01:27.236682 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a47ccca-0a4c-44cb-8fca-ffad5b1f08d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:27.238292 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a47ccca-0a4c-44cb-8fca-ffad5b1f08d6 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:27.251329 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1243/2499] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 23:01:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 23:01:27.257399 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47ccca-0a4c-44cb-8fca-ffad5b1f08d6 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:27.265906 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47ccca-0a4c-44cb-8fca-ffad5b1f08d6 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:27.271157 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b688d911-7300-409f-8e0f-ca6b04b12c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:27.271803 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b688d911-7300-409f-8e0f-ca6b04b12c28 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:27.272090 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b688d911-7300-409f-8e0f-ca6b04b12c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:27.272370 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b688d911-7300-409f-8e0f-ca6b04b12c28 admin admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:27.273053 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b688d911-7300-409f-8e0f-ca6b04b12c28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:27.274991 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b688d911-7300-409f-8e0f-ca6b04b12c28 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:27.369670 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1244/2500] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 23:01:27 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 23:01:27.370597 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b688d911-7300-409f-8e0f-ca6b04b12c28 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:27.424312 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-662b3756-245f-4c6b-ad59-eca4db985c1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd10c68b7fd4446feb759832913fea3b3'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:27.430421 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-662b3756-245f-4c6b-ad59-eca4db985c1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd10c68b7fd4446feb759832913fea3b3'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:27.431190 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-662b3756-245f-4c6b-ad59-eca4db985c1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd10c68b7fd4446feb759832913fea3b3'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:27.444267 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-662b3756-245f-4c6b-ad59-eca4db985c1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd10c68b7fd4446feb759832913fea3b3'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:27.454522 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b688d911-7300-409f-8e0f-ca6b04b12c28 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:27.459253 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3f273e-3b1c-4333-a1b2-105d7f6ede80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:27.459953 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f273e-3b1c-4333-a1b2-105d7f6ede80 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:27.460240 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f273e-3b1c-4333-a1b2-105d7f6ede80 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:27.460526 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f273e-3b1c-4333-a1b2-105d7f6ede80 admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:27.461207 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3f273e-3b1c-4333-a1b2-105d7f6ede80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:27.463145 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3f273e-3b1c-4333-a1b2-105d7f6ede80 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:27.526158 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-662b3756-245f-4c6b-ad59-eca4db985c1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd10c68b7fd4446feb759832913fea3b3'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:27.533081 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1257/2501] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 23:01:27 2022] DELETE /identity/v3/users/d10c68b7fd4446feb759832913fea3b3 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:01:27.562282 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1245/2502] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 23:01:27 2022] POST /identity/v3/users => generated 293 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:01:27.568698 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662b3756-245f-4c6b-ad59-eca4db985c1b admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:27.570252 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-662b3756-245f-4c6b-ad59-eca4db985c1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:27.654932 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662b3756-245f-4c6b-ad59-eca4db985c1b admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:27.662885 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bd0bb4-1056-4653-bd3a-8768d8722af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:27.662885 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bd0bb4-1056-4653-bd3a-8768d8722af6 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:27.662885 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bd0bb4-1056-4653-bd3a-8768d8722af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:27.662885 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bd0bb4-1056-4653-bd3a-8768d8722af6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:27.663334 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bd0bb4-1056-4653-bd3a-8768d8722af6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:27.664392 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bd0bb4-1056-4653-bd3a-8768d8722af6 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:27.676369 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1258/2503] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:01:27 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:01:27.680931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3f273e-3b1c-4333-a1b2-105d7f6ede80 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:27.690989 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3f273e-3b1c-4333-a1b2-105d7f6ede80 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:27.695638 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff809b-c14e-4d25-b518-d41e62c26046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:27.696323 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff809b-c14e-4d25-b518-d41e62c26046 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:27.696722 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff809b-c14e-4d25-b518-d41e62c26046 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:27.697007 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff809b-c14e-4d25-b518-d41e62c26046 admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:27.697734 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dff809b-c14e-4d25-b518-d41e62c26046 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:27.699371 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dff809b-c14e-4d25-b518-d41e62c26046 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:27.710494 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1246/2504] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:01:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:01:27.714649 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bd0bb4-1056-4653-bd3a-8768d8722af6 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:27.725359 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bd0bb4-1056-4653-bd3a-8768d8722af6 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:27.730644 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd9b784-4136-419d-baa9-ee9c8e9962a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:27.731573 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd9b784-4136-419d-baa9-ee9c8e9962a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:27.731977 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd9b784-4136-419d-baa9-ee9c8e9962a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:27.732252 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd9b784-4136-419d-baa9-ee9c8e9962a4 admin admin] PATH_INFO: `/v3/projects/8fe33afbf1594ff28ad628377258e4b8/users/c99f7e1403c74f69b517dc4b8ff3af7b/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:27.747837 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd9b784-4136-419d-baa9-ee9c8e9962a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fe33afbf1594ff28ad628377258e4b8, user_id=c99f7e1403c74f69b517dc4b8ff3af7b, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:27.750466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd9b784-4136-419d-baa9-ee9c8e9962a4 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:27.803198 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff809b-c14e-4d25-b518-d41e62c26046 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:27.805064 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-2dff809b-c14e-4d25-b518-d41e62c26046 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:27.821407 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff809b-c14e-4d25-b518-d41e62c26046 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:27.828322 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade84282-eafe-4fad-af8a-9761a554fb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:27.828322 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade84282-eafe-4fad-af8a-9761a554fb81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:27.828322 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade84282-eafe-4fad-af8a-9761a554fb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:27.828322 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade84282-eafe-4fad-af8a-9761a554fb81 admin admin] PATH_INFO: `/v3/projects/2f7e8103418741398872d59d16e0e919` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:27.829455 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade84282-eafe-4fad-af8a-9761a554fb81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f7e8103418741398872d59d16e0e919)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:27.832818 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade84282-eafe-4fad-af8a-9761a554fb81 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:27.834174 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1259/2505] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 23:01:27 2022] PUT /identity/v3/projects/8fe33afbf1594ff28ad628377258e4b8/users/c99f7e1403c74f69b517dc4b8ff3af7b/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:01:27.845810 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa72e83-92c9-430b-b239-f2ab2747d704 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:27.846318 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa72e83-92c9-430b-b239-f2ab2747d704 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:27.846835 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa72e83-92c9-430b-b239-f2ab2747d704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:27.860726 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:01:27.876167 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-9aa72e83-92c9-430b-b239-f2ab2747d704 None None] MFA Rules not processed for user `c99f7e1403c74f69b517dc4b8ff3af7b`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:01:27.877391 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-9aa72e83-92c9-430b-b239-f2ab2747d704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:27.944893 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1260/2506] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:01:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:01:27.949459 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa72e83-92c9-430b-b239-f2ab2747d704 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:28.009586 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-ade84282-eafe-4fad-af8a-9761a554fb81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f7e8103418741398872d59d16e0e919', 'request_id': 'req-ade84282-eafe-4fad-af8a-9761a554fb81'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:28.012604 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa72e83-92c9-430b-b239-f2ab2747d704 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:28.015195 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-ade84282-eafe-4fad-af8a-9761a554fb81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f7e8103418741398872d59d16e0e919', 'request_id': 'req-ade84282-eafe-4fad-af8a-9761a554fb81'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:28.069160 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d040374-ae45-4b5a-b9a1-bf24d252e211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:28.069823 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d040374-ae45-4b5a-b9a1-bf24d252e211 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:28.070106 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d040374-ae45-4b5a-b9a1-bf24d252e211 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:28.070382 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d040374-ae45-4b5a-b9a1-bf24d252e211 admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:28.071109 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d040374-ae45-4b5a-b9a1-bf24d252e211 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:28.073004 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d040374-ae45-4b5a-b9a1-bf24d252e211 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:28.112043 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-ade84282-eafe-4fad-af8a-9761a554fb81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f7e8103418741398872d59d16e0e919', 'request_id': 'req-ade84282-eafe-4fad-af8a-9761a554fb81'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:28.115096 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:01:28.125020 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-ade84282-eafe-4fad-af8a-9761a554fb81 admin admin] The token cache is being invalidate because project 2f7e8103418741398872d59d16e0e919 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:01:28.125307 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-ade84282-eafe-4fad-af8a-9761a554fb81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:28.125953 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-ade84282-eafe-4fad-af8a-9761a554fb81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:28.127569 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1247/2507] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 23:01:27 2022] DELETE /identity/v3/projects/2f7e8103418741398872d59d16e0e919 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 23:01:28.133245 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1708b1-888c-452e-ac93-96554bca8d1a None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:28.133617 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1708b1-888c-452e-ac93-96554bca8d1a None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:28.133983 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1708b1-888c-452e-ac93-96554bca8d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:28.155953 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-ad1708b1-888c-452e-ac93-96554bca8d1a None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:01:28.157169 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-ad1708b1-888c-452e-ac93-96554bca8d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:28.162248 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1261/2508] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 23:01:27 2022] POST /identity/v3/projects => generated 383 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 23:01:28.166445 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d040374-ae45-4b5a-b9a1-bf24d252e211 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:28.166872 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1248/2509] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:01:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 23:01:28.173440 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-8d040374-ae45-4b5a-b9a1-bf24d252e211 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:28.174467 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00165554-117e-455b-81cc-cccf07943585 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:28.174760 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00165554-117e-455b-81cc-cccf07943585 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:28.178257 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00165554-117e-455b-81cc-cccf07943585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:28.194499 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-00165554-117e-455b-81cc-cccf07943585 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:01:28.195792 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-00165554-117e-455b-81cc-cccf07943585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:28.211875 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:01:28.248932 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d040374-ae45-4b5a-b9a1-bf24d252e211 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:28.253694 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c41da7-569e-4820-9ff1-3b1a38b03562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:28.254379 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c41da7-569e-4820-9ff1-3b1a38b03562 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:28.255007 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c41da7-569e-4820-9ff1-3b1a38b03562 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:28.255350 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c41da7-569e-4820-9ff1-3b1a38b03562 admin admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:28.256460 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c41da7-569e-4820-9ff1-3b1a38b03562 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:28.259318 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c41da7-569e-4820-9ff1-3b1a38b03562 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:28.264206 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1249/2510] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:01:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 23:01:28.265864 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00165554-117e-455b-81cc-cccf07943585 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:28.276713 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00165554-117e-455b-81cc-cccf07943585 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:28.279611 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fd7014-247c-4358-bf71-b6c30665d587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:28.280277 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd7014-247c-4358-bf71-b6c30665d587 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:28.280557 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd7014-247c-4358-bf71-b6c30665d587 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:28.280834 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd7014-247c-4358-bf71-b6c30665d587 admin admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:28.281552 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fd7014-247c-4358-bf71-b6c30665d587 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 23:01:28.281854 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fd7014-247c-4358-bf71-b6c30665d587 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:28.283479 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fd7014-247c-4358-bf71-b6c30665d587 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:28.296521 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1250/2511] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 23:01:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 23:01:28.301010 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fd7014-247c-4358-bf71-b6c30665d587 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:28.314573 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fd7014-247c-4358-bf71-b6c30665d587 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:28.316304 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4883416e-c8cc-4f48-bfb1-c20d79176c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:28.317002 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4883416e-c8cc-4f48-bfb1-c20d79176c07 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:28.317776 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4883416e-c8cc-4f48-bfb1-c20d79176c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:28.318054 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4883416e-c8cc-4f48-bfb1-c20d79176c07 admin admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:28.318738 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4883416e-c8cc-4f48-bfb1-c20d79176c07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:28.320649 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4883416e-c8cc-4f48-bfb1-c20d79176c07 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:28.364035 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1262/2512] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 23:01:28 2022] POST /identity/v3/users => generated 298 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:01:28.367768 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c41da7-569e-4820-9ff1-3b1a38b03562 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:28.377356 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c41da7-569e-4820-9ff1-3b1a38b03562 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:28.383517 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e689e-ad9e-4c2b-b0c6-bf38d69503a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:28.390731 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e689e-ad9e-4c2b-b0c6-bf38d69503a5 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:28.390731 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e689e-ad9e-4c2b-b0c6-bf38d69503a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:28.390731 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e689e-ad9e-4c2b-b0c6-bf38d69503a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:28.390731 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364e689e-ad9e-4c2b-b0c6-bf38d69503a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:28.390731 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364e689e-ad9e-4c2b-b0c6-bf38d69503a5 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:28.398619 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1263/2513] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:01:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 23:01:28.402475 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e689e-ad9e-4c2b-b0c6-bf38d69503a5 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:28.413603 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e689e-ad9e-4c2b-b0c6-bf38d69503a5 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:28.419242 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa1beae-7c12-47df-a5fd-1ab2a7673179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:28.419400 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa1beae-7c12-47df-a5fd-1ab2a7673179 admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:28.419784 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa1beae-7c12-47df-a5fd-1ab2a7673179 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:28.420057 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa1beae-7c12-47df-a5fd-1ab2a7673179 admin admin] PATH_INFO: `/v3/projects/fdf95f1643c348c9a497390026b83eef/users/cb5fac5daaa0428d8ae033a8b339c3fc/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:28.420421 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1251/2514] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 23:01:28 2022] POST /identity/v3/projects => generated 367 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 23:01:28.425571 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4883416e-c8cc-4f48-bfb1-c20d79176c07 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:28.438822 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa1beae-7c12-47df-a5fd-1ab2a7673179 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdf95f1643c348c9a497390026b83eef, user_id=cb5fac5daaa0428d8ae033a8b339c3fc, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:28.442766 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa1beae-7c12-47df-a5fd-1ab2a7673179 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:28.506638 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4883416e-c8cc-4f48-bfb1-c20d79176c07 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:28.511369 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb98c6e4-f62f-464d-a38e-3d10414ffc66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:28.512045 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb98c6e4-f62f-464d-a38e-3d10414ffc66 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:28.512331 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb98c6e4-f62f-464d-a38e-3d10414ffc66 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:28.512608 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb98c6e4-f62f-464d-a38e-3d10414ffc66 admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:28.513291 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb98c6e4-f62f-464d-a38e-3d10414ffc66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:28.515254 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb98c6e4-f62f-464d-a38e-3d10414ffc66 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:28.528485 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1264/2515] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 23:01:28 2022] PUT /identity/v3/projects/fdf95f1643c348c9a497390026b83eef/users/cb5fac5daaa0428d8ae033a8b339c3fc/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 23:01:28.528485 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa1beae-7c12-47df-a5fd-1ab2a7673179 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:28.592466 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa1beae-7c12-47df-a5fd-1ab2a7673179 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:28.601211 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1252/2516] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 23:01:28 2022] POST /identity/v3/users => generated 284 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 23:01:28.601211 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920309b-8794-41b1-b6c7-227150a0d960 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:28.601211 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920309b-8794-41b1-b6c7-227150a0d960 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:28.601211 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920309b-8794-41b1-b6c7-227150a0d960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:28.601657 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef3d94-4394-41cf-b7e5-aca69fcdab91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:28.601657 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef3d94-4394-41cf-b7e5-aca69fcdab91 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:28.601657 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef3d94-4394-41cf-b7e5-aca69fcdab91 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:28.601657 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef3d94-4394-41cf-b7e5-aca69fcdab91 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:28.601997 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ef3d94-4394-41cf-b7e5-aca69fcdab91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:28.601997 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ef3d94-4394-41cf-b7e5-aca69fcdab91 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:28.611854 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1265/2517] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:01:28 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:01:28.616652 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef3d94-4394-41cf-b7e5-aca69fcdab91 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:28.623708 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef3d94-4394-41cf-b7e5-aca69fcdab91 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:28.628547 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba59af5d-d63c-4b8d-aa31-660689ce0709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:28.629216 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba59af5d-d63c-4b8d-aa31-660689ce0709 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:28.629498 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba59af5d-d63c-4b8d-aa31-660689ce0709 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:28.630457 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba59af5d-d63c-4b8d-aa31-660689ce0709 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:28.631227 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba59af5d-d63c-4b8d-aa31-660689ce0709 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:28.631341 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-f920309b-8794-41b1-b6c7-227150a0d960 None None] MFA Rules not processed for user `63cc904578554aea84ff1917a5ed075e`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:01:28.632545 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-f920309b-8794-41b1-b6c7-227150a0d960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:28.632860 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba59af5d-d63c-4b8d-aa31-660689ce0709 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:28.643439 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1266/2518] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:01:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:01:28.644364 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba59af5d-d63c-4b8d-aa31-660689ce0709 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:28.656199 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba59af5d-d63c-4b8d-aa31-660689ce0709 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:28.661325 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f85e11d-aac3-4623-b80a-7dd746e76861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:28.662367 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85e11d-aac3-4623-b80a-7dd746e76861 admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:28.662729 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85e11d-aac3-4623-b80a-7dd746e76861 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:28.663110 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85e11d-aac3-4623-b80a-7dd746e76861 admin admin] PATH_INFO: `/v3/projects/fdf95f1643c348c9a497390026b83eef/users/cb5fac5daaa0428d8ae033a8b339c3fc/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:28.665234 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f85e11d-aac3-4623-b80a-7dd746e76861 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdf95f1643c348c9a497390026b83eef, user_id=cb5fac5daaa0428d8ae033a8b339c3fc, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:28.666879 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:01:28.667939 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f85e11d-aac3-4623-b80a-7dd746e76861 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:28.681327 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1267/2519] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 23:01:28 2022] PUT /identity/v3/projects/fdf95f1643c348c9a497390026b83eef/users/cb5fac5daaa0428d8ae033a8b339c3fc/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:01:28.687711 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f85e11d-aac3-4623-b80a-7dd746e76861 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:28.699221 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1253/2520] 10.222.0.52 () {58 vars in 967 bytes} [Thu Jul 21 23:01:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:01:28.700645 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3748d3d8-8a30-4b7a-a3b5-ed4569a5b868 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:28.700989 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3748d3d8-8a30-4b7a-a3b5-ed4569a5b868 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:28.701368 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3748d3d8-8a30-4b7a-a3b5-ed4569a5b868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:28.730434 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-3748d3d8-8a30-4b7a-a3b5-ed4569a5b868 None None] MFA Rules not processed for user `cb5fac5daaa0428d8ae033a8b339c3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:01:28.731880 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-3748d3d8-8a30-4b7a-a3b5-ed4569a5b868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:28.751561 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:01:28.757327 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f85e11d-aac3-4623-b80a-7dd746e76861 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:28.763928 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a9f1a-5546-48cc-bfef-1e9675e59d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:28.765901 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a9f1a-5546-48cc-bfef-1e9675e59d89 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:28.765901 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a9f1a-5546-48cc-bfef-1e9675e59d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:28.765901 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a9f1a-5546-48cc-bfef-1e9675e59d89 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:28.765901 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381a9f1a-5546-48cc-bfef-1e9675e59d89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:28.767665 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381a9f1a-5546-48cc-bfef-1e9675e59d89 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:28.782075 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1268/2521] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:01:28 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:01:28.782075 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a9f1a-5546-48cc-bfef-1e9675e59d89 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:28.782645 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-381a9f1a-5546-48cc-bfef-1e9675e59d89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:28.809159 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1254/2522] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:01:28 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:01:28.810280 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3748d3d8-8a30-4b7a-a3b5-ed4569a5b868 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:28.823130 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3748d3d8-8a30-4b7a-a3b5-ed4569a5b868 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:28.828166 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b7bef2-bb16-4d58-8631-8a96a3a171b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:28.829047 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7bef2-bb16-4d58-8631-8a96a3a171b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:28.829333 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7bef2-bb16-4d58-8631-8a96a3a171b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:28.829720 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7bef2-bb16-4d58-8631-8a96a3a171b8 admin admin] PATH_INFO: `/v3/projects/3028fa75183d4df4a9aa9adb506b56aa/users/7dea413fb6f94ba8bbd6f86ef68a999e/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:28.845928 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b7bef2-bb16-4d58-8631-8a96a3a171b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3028fa75183d4df4a9aa9adb506b56aa, user_id=7dea413fb6f94ba8bbd6f86ef68a999e, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:28.849309 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b7bef2-bb16-4d58-8631-8a96a3a171b8 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:28.863002 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a9f1a-5546-48cc-bfef-1e9675e59d89 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:28.868489 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677a5d33-ac7f-4dde-9a2d-01d2a5d9471d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba7eef4e1b27476ab4016449d9c3a4b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:28.869130 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a5d33-ac7f-4dde-9a2d-01d2a5d9471d service swift] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:28.869412 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a5d33-ac7f-4dde-9a2d-01d2a5d9471d service swift] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:28.869685 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a5d33-ac7f-4dde-9a2d-01d2a5d9471d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:28.932430 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677a5d33-ac7f-4dde-9a2d-01d2a5d9471d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:28.934076 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677a5d33-ac7f-4dde-9a2d-01d2a5d9471d service swift] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:28.942158 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1269/2523] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Jul 21 23:01:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 23:01:28.943601 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677a5d33-ac7f-4dde-9a2d-01d2a5d9471d service swift] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:28.944999 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-677a5d33-ac7f-4dde-9a2d-01d2a5d9471d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:28.953964 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1255/2524] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 23:01:28 2022] PUT /identity/v3/projects/3028fa75183d4df4a9aa9adb506b56aa/users/7dea413fb6f94ba8bbd6f86ef68a999e/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:01:28.969072 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1d394-2eff-4ae7-b5f5-8afe081dc695 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:28.969360 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1d394-2eff-4ae7-b5f5-8afe081dc695 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:28.969662 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1d394-2eff-4ae7-b5f5-8afe081dc695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:28.998219 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-d4f1d394-2eff-4ae7-b5f5-8afe081dc695 None None] MFA Rules not processed for user `7dea413fb6f94ba8bbd6f86ef68a999e`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:01:28.999482 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-d4f1d394-2eff-4ae7-b5f5-8afe081dc695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:29.005818 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677a5d33-ac7f-4dde-9a2d-01d2a5d9471d service swift] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:29.058282 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:01:29.063276 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38fa93e-5e96-4e58-ad67-18ad351ddc4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:29.064023 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38fa93e-5e96-4e58-ad67-18ad351ddc4f service cinder] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:29.064484 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38fa93e-5e96-4e58-ad67-18ad351ddc4f service cinder] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:29.064949 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38fa93e-5e96-4e58-ad67-18ad351ddc4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:29.067010 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-f38fa93e-5e96-4e58-ad67-18ad351ddc4f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:29.069703 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1256/2525] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:01:28 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:01:29.080818 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f1d394-2eff-4ae7-b5f5-8afe081dc695 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:29.082606 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-d4f1d394-2eff-4ae7-b5f5-8afe081dc695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:29.096601 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f1d394-2eff-4ae7-b5f5-8afe081dc695 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:29.104765 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f867b2-80f1-4095-809f-ac6f988885f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:29.105433 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f867b2-80f1-4095-809f-ac6f988885f0 service cinder] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:29.105722 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f867b2-80f1-4095-809f-ac6f988885f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:29.105997 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f867b2-80f1-4095-809f-ac6f988885f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:29.115960 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f867b2-80f1-4095-809f-ac6f988885f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:29.117609 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f867b2-80f1-4095-809f-ac6f988885f0 service cinder] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:29.127194 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1257/2526] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:01:29 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 23:01:29.144050 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38fa93e-5e96-4e58-ad67-18ad351ddc4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:29.144523 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38fa93e-5e96-4e58-ad67-18ad351ddc4f service cinder] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:29.154594 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1270/2527] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:01:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:01:32.430282 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f867b2-80f1-4095-809f-ac6f988885f0 service cinder] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:32.431724 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-75f867b2-80f1-4095-809f-ac6f988885f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:32.455541 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:01:32.502473 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f867b2-80f1-4095-809f-ac6f988885f0 service cinder] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:32.507220 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62ac3bd-a123-405b-80c3-9efe0cf25e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:32.508163 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ac3bd-a123-405b-80c3-9efe0cf25e8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:32.508447 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ac3bd-a123-405b-80c3-9efe0cf25e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:32.508726 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ac3bd-a123-405b-80c3-9efe0cf25e8b admin admin] PATH_INFO: `/v3/users/932c6d0ae774493a852781e4c8d83e67` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:32.510003 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62ac3bd-a123-405b-80c3-9efe0cf25e8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=932c6d0ae774493a852781e4c8d83e67)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:32.512042 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62ac3bd-a123-405b-80c3-9efe0cf25e8b admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:32.737260 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-a62ac3bd-a123-405b-80c3-9efe0cf25e8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '932c6d0ae774493a852781e4c8d83e67'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:32.746958 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-a62ac3bd-a123-405b-80c3-9efe0cf25e8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '932c6d0ae774493a852781e4c8d83e67'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:32.746958 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-a62ac3bd-a123-405b-80c3-9efe0cf25e8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '932c6d0ae774493a852781e4c8d83e67'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:32.758418 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-a62ac3bd-a123-405b-80c3-9efe0cf25e8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '932c6d0ae774493a852781e4c8d83e67'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:32.759145 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-a62ac3bd-a123-405b-80c3-9efe0cf25e8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '932c6d0ae774493a852781e4c8d83e67'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:32.837092 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1258/2528] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 23:01:32 2022] DELETE /identity/v3/users/932c6d0ae774493a852781e4c8d83e67 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:01:33.099735 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38fa93e-5e96-4e58-ad67-18ad351ddc4f service cinder] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:33.112455 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38fa93e-5e96-4e58-ad67-18ad351ddc4f service cinder] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:33.185319 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9d579d-a5b4-459d-81f6-51fc3cd205c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:33.186182 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d579d-a5b4-459d-81f6-51fc3cd205c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:33.186463 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d579d-a5b4-459d-81f6-51fc3cd205c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:33.186739 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d579d-a5b4-459d-81f6-51fc3cd205c1 admin admin] PATH_INFO: `/v3/projects/3429139a6e494a7290d91dd3cb8f712c` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:33.187978 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9d579d-a5b4-459d-81f6-51fc3cd205c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3429139a6e494a7290d91dd3cb8f712c)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:33.189887 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9d579d-a5b4-459d-81f6-51fc3cd205c1 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:33.307619 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-ce9d579d-a5b4-459d-81f6-51fc3cd205c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3429139a6e494a7290d91dd3cb8f712c', 'request_id': 'req-ce9d579d-a5b4-459d-81f6-51fc3cd205c1'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:33.313138 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-ce9d579d-a5b4-459d-81f6-51fc3cd205c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3429139a6e494a7290d91dd3cb8f712c', 'request_id': 'req-ce9d579d-a5b4-459d-81f6-51fc3cd205c1'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:33.418500 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-ce9d579d-a5b4-459d-81f6-51fc3cd205c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3429139a6e494a7290d91dd3cb8f712c', 'request_id': 'req-ce9d579d-a5b4-459d-81f6-51fc3cd205c1'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:33.431919 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-ce9d579d-a5b4-459d-81f6-51fc3cd205c1 admin admin] The token cache is being invalidate because project 3429139a6e494a7290d91dd3cb8f712c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:01:33.432314 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-ce9d579d-a5b4-459d-81f6-51fc3cd205c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:33.433016 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-ce9d579d-a5b4-459d-81f6-51fc3cd205c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:33.435011 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1271/2529] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 23:01:33 2022] DELETE /identity/v3/projects/3429139a6e494a7290d91dd3cb8f712c => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:01:33.442088 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba956d-f64a-4ae2-a9e5-f648da4d98f9 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:33.442626 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba956d-f64a-4ae2-a9e5-f648da4d98f9 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:33.442722 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba956d-f64a-4ae2-a9e5-f648da4d98f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:33.461807 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-4aba956d-f64a-4ae2-a9e5-f648da4d98f9 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:01:33.462990 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-4aba956d-f64a-4ae2-a9e5-f648da4d98f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:33.550618 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1259/2530] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:01:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:01:33.558698 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb7a1b-309f-4af9-9176-2bfed0a5900e None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:33.559019 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb7a1b-309f-4af9-9176-2bfed0a5900e None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:33.559326 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb7a1b-309f-4af9-9176-2bfed0a5900e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:33.590835 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-60fb7a1b-309f-4af9-9176-2bfed0a5900e None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:01:33.596127 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-60fb7a1b-309f-4af9-9176-2bfed0a5900e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:33.600602 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1272/2531] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:01:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:01:33.605039 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba956d-f64a-4ae2-a9e5-f648da4d98f9 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:33.615333 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba956d-f64a-4ae2-a9e5-f648da4d98f9 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:33.627302 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e80264-7d30-4708-b73a-8ca1098ede8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:33.627302 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e80264-7d30-4708-b73a-8ca1098ede8c admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:33.627302 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e80264-7d30-4708-b73a-8ca1098ede8c admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:33.627302 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e80264-7d30-4708-b73a-8ca1098ede8c admin admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:33.627679 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e80264-7d30-4708-b73a-8ca1098ede8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 23:01:33.627679 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e80264-7d30-4708-b73a-8ca1098ede8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:33.627679 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e80264-7d30-4708-b73a-8ca1098ede8c admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:33.640863 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1260/2532] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 23:01:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 23:01:33.645142 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb7a1b-309f-4af9-9176-2bfed0a5900e None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:33.654759 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb7a1b-309f-4af9-9176-2bfed0a5900e None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:33.659740 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fadc033-8501-4b96-bae2-7875e6a7edc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:33.660399 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fadc033-8501-4b96-bae2-7875e6a7edc9 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:33.660683 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fadc033-8501-4b96-bae2-7875e6a7edc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:33.660963 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fadc033-8501-4b96-bae2-7875e6a7edc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:33.661736 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fadc033-8501-4b96-bae2-7875e6a7edc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:33.664142 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fadc033-8501-4b96-bae2-7875e6a7edc9 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:33.805868 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1273/2533] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 23:01:33 2022] POST /identity/v3/projects => generated 371 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 23:01:33.810009 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e80264-7d30-4708-b73a-8ca1098ede8c admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:33.820699 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e80264-7d30-4708-b73a-8ca1098ede8c admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:33.842024 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:01:33.884050 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e7104-8d72-42e2-8fcb-23504f42dedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:33.884754 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e7104-8d72-42e2-8fcb-23504f42dedc admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:33.885041 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e7104-8d72-42e2-8fcb-23504f42dedc admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:33.885321 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e7104-8d72-42e2-8fcb-23504f42dedc admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:33.886016 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2e7104-8d72-42e2-8fcb-23504f42dedc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:33.887958 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2e7104-8d72-42e2-8fcb-23504f42dedc admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:33.984008 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1261/2534] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 23:01:33 2022] POST /identity/v3/users => generated 286 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:01:33.988023 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fadc033-8501-4b96-bae2-7875e6a7edc9 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:33.995446 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:01:33.999256 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fadc033-8501-4b96-bae2-7875e6a7edc9 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:34.004120 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8aa527-d880-4547-a7a0-a08b30a1a1aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:34.004837 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8aa527-d880-4547-a7a0-a08b30a1a1aa admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:34.005130 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8aa527-d880-4547-a7a0-a08b30a1a1aa admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:34.005407 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8aa527-d880-4547-a7a0-a08b30a1a1aa admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:34.006910 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8aa527-d880-4547-a7a0-a08b30a1a1aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:34.009290 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8aa527-d880-4547-a7a0-a08b30a1a1aa admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:34.031038 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1274/2535] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:01:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:01:34.031164 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e7104-8d72-42e2-8fcb-23504f42dedc admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:34.041762 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e7104-8d72-42e2-8fcb-23504f42dedc admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:34.046766 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad33aca-ff21-4e57-9cef-221f52be4876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:34.047452 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad33aca-ff21-4e57-9cef-221f52be4876 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:34.047740 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad33aca-ff21-4e57-9cef-221f52be4876 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:34.048243 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad33aca-ff21-4e57-9cef-221f52be4876 admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:34.049158 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad33aca-ff21-4e57-9cef-221f52be4876 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:34.051099 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad33aca-ff21-4e57-9cef-221f52be4876 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:34.064470 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1262/2536] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:01:34.069479 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8aa527-d880-4547-a7a0-a08b30a1a1aa admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:34.080165 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8aa527-d880-4547-a7a0-a08b30a1a1aa admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:34.084964 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d1f4e-3d9c-4b7b-8020-87818266e0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:34.085872 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d1f4e-3d9c-4b7b-8020-87818266e0ee admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:34.086155 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d1f4e-3d9c-4b7b-8020-87818266e0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:34.086473 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d1f4e-3d9c-4b7b-8020-87818266e0ee admin admin] PATH_INFO: `/v3/projects/a0e5f01e7bae402d9ee2697ff9ae5461/users/e3b076c33f784569a1fafd49f7c2f314/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:34.101995 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71d1f4e-3d9c-4b7b-8020-87818266e0ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0e5f01e7bae402d9ee2697ff9ae5461, user_id=e3b076c33f784569a1fafd49f7c2f314, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:34.104965 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71d1f4e-3d9c-4b7b-8020-87818266e0ee admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:34.179290 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1275/2537] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 23:01:34 2022] PUT /identity/v3/projects/a0e5f01e7bae402d9ee2697ff9ae5461/users/e3b076c33f784569a1fafd49f7c2f314/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:01:34.189788 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401bd9c-8219-49cf-93a5-b58f1530d387 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:34.190080 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401bd9c-8219-49cf-93a5-b58f1530d387 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:34.190359 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401bd9c-8219-49cf-93a5-b58f1530d387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:34.218160 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-e401bd9c-8219-49cf-93a5-b58f1530d387 None None] MFA Rules not processed for user `e3b076c33f784569a1fafd49f7c2f314`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:01:34.219588 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-e401bd9c-8219-49cf-93a5-b58f1530d387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:34.290970 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1263/2538] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:01:34 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:01:34.302935 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d1f4e-3d9c-4b7b-8020-87818266e0ee admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:34.304685 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-e71d1f4e-3d9c-4b7b-8020-87818266e0ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:34.367953 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d1f4e-3d9c-4b7b-8020-87818266e0ee admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:34.371664 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80e4ca9-3117-44a4-b859-e8fdf72adf6b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '927778d005fd48a4b4b8fd3c4419bb36', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:34.372364 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e4ca9-3117-44a4-b859-e8fdf72adf6b service glance] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:34.372684 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e4ca9-3117-44a4-b859-e8fdf72adf6b service glance] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:34.372962 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e4ca9-3117-44a4-b859-e8fdf72adf6b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:34.382197 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80e4ca9-3117-44a4-b859-e8fdf72adf6b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:34.383665 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80e4ca9-3117-44a4-b859-e8fdf72adf6b service glance] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:34.392623 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1276/2539] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:01:34 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:01:34.882361 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e401bd9c-8219-49cf-93a5-b58f1530d387 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:34.884843 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-e401bd9c-8219-49cf-93a5-b58f1530d387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:34.975029 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e401bd9c-8219-49cf-93a5-b58f1530d387 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:34.978251 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7669afd6-f0ed-44ec-8648-457a6849f20c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:34.979213 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669afd6-f0ed-44ec-8648-457a6849f20c service cinder] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:34.979503 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669afd6-f0ed-44ec-8648-457a6849f20c service cinder] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:34.979788 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669afd6-f0ed-44ec-8648-457a6849f20c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:34.981676 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-7669afd6-f0ed-44ec-8648-457a6849f20c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:35.009992 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:01:35.046543 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7669afd6-f0ed-44ec-8648-457a6849f20c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:35.047927 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7669afd6-f0ed-44ec-8648-457a6849f20c service cinder] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:35.057718 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1264/2540] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:01:34 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:01:40.069927 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80e4ca9-3117-44a4-b859-e8fdf72adf6b service glance] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:40.071467 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-f80e4ca9-3117-44a4-b859-e8fdf72adf6b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:40.139136 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80e4ca9-3117-44a4-b859-e8fdf72adf6b service glance] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:40.143728 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d381ec03-98c9-45d9-8e2c-ef822f99813f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:40.144596 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381ec03-98c9-45d9-8e2c-ef822f99813f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:40.144882 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381ec03-98c9-45d9-8e2c-ef822f99813f admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:40.145160 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381ec03-98c9-45d9-8e2c-ef822f99813f admin admin] PATH_INFO: `/v3/users/7dea413fb6f94ba8bbd6f86ef68a999e` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:40.146374 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d381ec03-98c9-45d9-8e2c-ef822f99813f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dea413fb6f94ba8bbd6f86ef68a999e)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:40.148946 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d381ec03-98c9-45d9-8e2c-ef822f99813f admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:40.343598 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-d381ec03-98c9-45d9-8e2c-ef822f99813f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dea413fb6f94ba8bbd6f86ef68a999e'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:40.350422 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-d381ec03-98c9-45d9-8e2c-ef822f99813f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dea413fb6f94ba8bbd6f86ef68a999e'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:40.351146 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-d381ec03-98c9-45d9-8e2c-ef822f99813f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dea413fb6f94ba8bbd6f86ef68a999e'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:40.363019 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-d381ec03-98c9-45d9-8e2c-ef822f99813f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dea413fb6f94ba8bbd6f86ef68a999e'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:40.364676 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:01:40.461051 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-d381ec03-98c9-45d9-8e2c-ef822f99813f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dea413fb6f94ba8bbd6f86ef68a999e'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:40.467569 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1277/2541] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 23:01:40 2022] DELETE /identity/v3/users/7dea413fb6f94ba8bbd6f86ef68a999e => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:01:40.472467 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef21517-bce1-4e0a-8c97-495a916115b1 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:40.472966 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef21517-bce1-4e0a-8c97-495a916115b1 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:40.473065 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef21517-bce1-4e0a-8c97-495a916115b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:40.493828 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-2ef21517-bce1-4e0a-8c97-495a916115b1 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:01:40.495396 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-2ef21517-bce1-4e0a-8c97-495a916115b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:40.558283 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1265/2542] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:01:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:01:40.568280 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d381ec03-98c9-45d9-8e2c-ef822f99813f admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:40.570335 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-d381ec03-98c9-45d9-8e2c-ef822f99813f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:40.665456 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d381ec03-98c9-45d9-8e2c-ef822f99813f admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:40.669180 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b39d621-57c9-4c5b-a792-3699e076ef6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:40.669858 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b39d621-57c9-4c5b-a792-3699e076ef6f service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:40.670138 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b39d621-57c9-4c5b-a792-3699e076ef6f service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:40.670413 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b39d621-57c9-4c5b-a792-3699e076ef6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:40.680101 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b39d621-57c9-4c5b-a792-3699e076ef6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:40.681530 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b39d621-57c9-4c5b-a792-3699e076ef6f service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:40.693292 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1278/2543] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:01:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:01:41.104218 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef21517-bce1-4e0a-8c97-495a916115b1 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:41.105685 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-2ef21517-bce1-4e0a-8c97-495a916115b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:41.122041 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef21517-bce1-4e0a-8c97-495a916115b1 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:41.130487 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31515b5-a588-4759-a2e6-a99eaffea956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:41.131768 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31515b5-a588-4759-a2e6-a99eaffea956 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:41.132098 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31515b5-a588-4759-a2e6-a99eaffea956 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:41.132415 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31515b5-a588-4759-a2e6-a99eaffea956 admin admin] PATH_INFO: `/v3/projects/3028fa75183d4df4a9aa9adb506b56aa` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:41.134581 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31515b5-a588-4759-a2e6-a99eaffea956 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3028fa75183d4df4a9aa9adb506b56aa)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:41.139057 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31515b5-a588-4759-a2e6-a99eaffea956 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:41.282890 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-a31515b5-a588-4759-a2e6-a99eaffea956 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3028fa75183d4df4a9aa9adb506b56aa', 'request_id': 'req-a31515b5-a588-4759-a2e6-a99eaffea956'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:41.289047 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-a31515b5-a588-4759-a2e6-a99eaffea956 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3028fa75183d4df4a9aa9adb506b56aa', 'request_id': 'req-a31515b5-a588-4759-a2e6-a99eaffea956'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:41.335027 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b39d621-57c9-4c5b-a792-3699e076ef6f service neutron] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:41.336481 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-1b39d621-57c9-4c5b-a792-3699e076ef6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:41.349336 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b39d621-57c9-4c5b-a792-3699e076ef6f service neutron] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:41.363788 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-a31515b5-a588-4759-a2e6-a99eaffea956 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3028fa75183d4df4a9aa9adb506b56aa', 'request_id': 'req-a31515b5-a588-4759-a2e6-a99eaffea956'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:41.379851 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-a31515b5-a588-4759-a2e6-a99eaffea956 admin admin] The token cache is being invalidate because project 3028fa75183d4df4a9aa9adb506b56aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:01:41.380141 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-a31515b5-a588-4759-a2e6-a99eaffea956 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:41.380891 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-a31515b5-a588-4759-a2e6-a99eaffea956 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:41.382644 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1266/2544] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 23:01:41 2022] DELETE /identity/v3/projects/3028fa75183d4df4a9aa9adb506b56aa => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:01:41.387593 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c282a-0d4b-446f-9abb-1c0f8652e3ae None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:41.387888 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c282a-0d4b-446f-9abb-1c0f8652e3ae None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:41.388169 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c282a-0d4b-446f-9abb-1c0f8652e3ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:41.407008 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-de4c282a-0d4b-446f-9abb-1c0f8652e3ae None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:01:41.408227 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-de4c282a-0d4b-446f-9abb-1c0f8652e3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:41.411685 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c27284-d8d9-413e-ab9d-3baf31ea9a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:41.413136 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c27284-d8d9-413e-ab9d-3baf31ea9a81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:41.413400 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c27284-d8d9-413e-ab9d-3baf31ea9a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:41.413727 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c27284-d8d9-413e-ab9d-3baf31ea9a81 admin admin] PATH_INFO: `/v3/users/c99f7e1403c74f69b517dc4b8ff3af7b` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:41.414886 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c27284-d8d9-413e-ab9d-3baf31ea9a81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c99f7e1403c74f69b517dc4b8ff3af7b)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:41.416794 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c27284-d8d9-413e-ab9d-3baf31ea9a81 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:41.417261 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1267/2545] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:01:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 23:01:41.425056 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff398ba-9627-41cd-8dd5-48c2d18d94f3 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:41.425413 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff398ba-9627-41cd-8dd5-48c2d18d94f3 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:41.425692 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff398ba-9627-41cd-8dd5-48c2d18d94f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:41.445723 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-1ff398ba-9627-41cd-8dd5-48c2d18d94f3 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:01:41.447013 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-1ff398ba-9627-41cd-8dd5-48c2d18d94f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:41.455729 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1268/2546] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:01:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 23:01:41.460721 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff398ba-9627-41cd-8dd5-48c2d18d94f3 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:41.469907 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff398ba-9627-41cd-8dd5-48c2d18d94f3 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:41.477323 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6105f549-5c6e-4093-8cfd-774222f5d708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:41.477323 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6105f549-5c6e-4093-8cfd-774222f5d708 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:41.477323 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6105f549-5c6e-4093-8cfd-774222f5d708 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:41.477323 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6105f549-5c6e-4093-8cfd-774222f5d708 admin admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:41.478156 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6105f549-5c6e-4093-8cfd-774222f5d708 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 23:01:41.478156 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6105f549-5c6e-4093-8cfd-774222f5d708 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:41.479222 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6105f549-5c6e-4093-8cfd-774222f5d708 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:41.494941 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1269/2547] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 23:01:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 23:01:41.496029 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6105f549-5c6e-4093-8cfd-774222f5d708 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:41.507090 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6105f549-5c6e-4093-8cfd-774222f5d708 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:41.512105 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84357564-8db4-48e8-85ac-0fef3768d735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:41.512778 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84357564-8db4-48e8-85ac-0fef3768d735 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:41.513067 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84357564-8db4-48e8-85ac-0fef3768d735 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:41.513347 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84357564-8db4-48e8-85ac-0fef3768d735 admin admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:41.514050 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84357564-8db4-48e8-85ac-0fef3768d735 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:41.516131 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84357564-8db4-48e8-85ac-0fef3768d735 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:41.604513 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:01:41.612914 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1270/2548] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 23:01:41 2022] POST /identity/v3/projects => generated 405 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:01:41.615488 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84357564-8db4-48e8-85ac-0fef3768d735 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:41.655578 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-a6c27284-d8d9-413e-ab9d-3baf31ea9a81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c99f7e1403c74f69b517dc4b8ff3af7b'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:41.663287 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-a6c27284-d8d9-413e-ab9d-3baf31ea9a81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c99f7e1403c74f69b517dc4b8ff3af7b'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:41.664014 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-a6c27284-d8d9-413e-ab9d-3baf31ea9a81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c99f7e1403c74f69b517dc4b8ff3af7b'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:41.678064 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-a6c27284-d8d9-413e-ab9d-3baf31ea9a81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c99f7e1403c74f69b517dc4b8ff3af7b'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:41.680661 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:01:41.683551 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84357564-8db4-48e8-85ac-0fef3768d735 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:41.692362 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b714628b-fce7-4e81-ad3b-ba14cb2b58d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:41.692362 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714628b-fce7-4e81-ad3b-ba14cb2b58d5 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:41.692362 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714628b-fce7-4e81-ad3b-ba14cb2b58d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:41.692362 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714628b-fce7-4e81-ad3b-ba14cb2b58d5 admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:41.692737 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b714628b-fce7-4e81-ad3b-ba14cb2b58d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:41.692737 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b714628b-fce7-4e81-ad3b-ba14cb2b58d5 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:41.769152 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-a6c27284-d8d9-413e-ab9d-3baf31ea9a81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c99f7e1403c74f69b517dc4b8ff3af7b'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:41.776331 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1279/2549] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 23:01:41 2022] DELETE /identity/v3/users/c99f7e1403c74f69b517dc4b8ff3af7b => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:01:41.780648 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c00b98-be48-4667-a4a1-6e4fb312d7b3 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:41.780941 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c00b98-be48-4667-a4a1-6e4fb312d7b3 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:41.781330 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c00b98-be48-4667-a4a1-6e4fb312d7b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:41.799542 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-06c00b98-be48-4667-a4a1-6e4fb312d7b3 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:01:41.801991 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-06c00b98-be48-4667-a4a1-6e4fb312d7b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:41.806934 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1271/2550] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 23:01:41 2022] POST /identity/v3/users => generated 303 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 23:01:41.808378 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b714628b-fce7-4e81-ad3b-ba14cb2b58d5 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:41.810113 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-b714628b-fce7-4e81-ad3b-ba14cb2b58d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:41.822977 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b714628b-fce7-4e81-ad3b-ba14cb2b58d5 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:41.882502 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1280/2551] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:01:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 23:01:41.900617 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c00b98-be48-4667-a4a1-6e4fb312d7b3 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:41.910919 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-06c00b98-be48-4667-a4a1-6e4fb312d7b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:41.911029 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08c179b-6131-4987-964d-ae47aff66a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:41.911029 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c179b-6131-4987-964d-ae47aff66a1c admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:41.911029 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c179b-6131-4987-964d-ae47aff66a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:41.911029 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c179b-6131-4987-964d-ae47aff66a1c admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:41.911423 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08c179b-6131-4987-964d-ae47aff66a1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:41.912698 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08c179b-6131-4987-964d-ae47aff66a1c admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:41.927431 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1272/2552] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:01:41 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:01:41.928502 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08c179b-6131-4987-964d-ae47aff66a1c admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:41.939244 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08c179b-6131-4987-964d-ae47aff66a1c admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:41.946660 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaeef87-3bf3-4ec3-9d81-f4347818b35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:41.947361 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaeef87-3bf3-4ec3-9d81-f4347818b35b admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:41.948383 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaeef87-3bf3-4ec3-9d81-f4347818b35b admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:41.948383 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaeef87-3bf3-4ec3-9d81-f4347818b35b admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:41.948659 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaeef87-3bf3-4ec3-9d81-f4347818b35b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:41.950250 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaeef87-3bf3-4ec3-9d81-f4347818b35b admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:41.964401 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1273/2553] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:01:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 23:01:41.965742 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaeef87-3bf3-4ec3-9d81-f4347818b35b admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:41.976253 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaeef87-3bf3-4ec3-9d81-f4347818b35b admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:41.981139 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dcbedc-2a9e-447f-9894-428812d2fcab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:41.982022 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dcbedc-2a9e-447f-9894-428812d2fcab admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:41.982330 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dcbedc-2a9e-447f-9894-428812d2fcab admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:41.982609 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dcbedc-2a9e-447f-9894-428812d2fcab admin admin] PATH_INFO: `/v3/projects/fe4174e900e542c992af44c90f4e7fd2/users/7146cf2476be44c6ab118e8737cdbb08/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:41.993429 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c00b98-be48-4667-a4a1-6e4fb312d7b3 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:42.000251 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3bc431-b1c8-445b-8459-f0b2f57937c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:42.000700 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dcbedc-2a9e-447f-9894-428812d2fcab admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe4174e900e542c992af44c90f4e7fd2, user_id=7146cf2476be44c6ab118e8737cdbb08, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:42.000940 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3bc431-b1c8-445b-8459-f0b2f57937c3 service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:42.001220 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3bc431-b1c8-445b-8459-f0b2f57937c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:42.001492 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3bc431-b1c8-445b-8459-f0b2f57937c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:42.003557 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dcbedc-2a9e-447f-9894-428812d2fcab admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:42.011750 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3bc431-b1c8-445b-8459-f0b2f57937c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:42.013194 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3bc431-b1c8-445b-8459-f0b2f57937c3 service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:42.022654 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1281/2554] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:01:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:01:42.024191 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c309b9f9-e5f0-4798-91ca-df2753209da3 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:42.024476 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c309b9f9-e5f0-4798-91ca-df2753209da3 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:42.024752 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c309b9f9-e5f0-4798-91ca-df2753209da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:42.044075 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-c309b9f9-e5f0-4798-91ca-df2753209da3 None None] MFA Rules not processed for user `ae2a190cf2ef4777a3838cbe9750afea`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:01:42.045830 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-c309b9f9-e5f0-4798-91ca-df2753209da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:42.102082 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1274/2555] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 23:01:41 2022] PUT /identity/v3/projects/fe4174e900e542c992af44c90f4e7fd2/users/7146cf2476be44c6ab118e8737cdbb08/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 23:01:42.107708 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46da9f-9885-4de3-b89d-709407128990 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:42.108118 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46da9f-9885-4de3-b89d-709407128990 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:42.108562 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46da9f-9885-4de3-b89d-709407128990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:42.117582 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1282/2556] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:01:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:01:42.128459 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-2a46da9f-9885-4de3-b89d-709407128990 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:01:42.134527 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-2a46da9f-9885-4de3-b89d-709407128990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:42.182492 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:01:42.200149 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1275/2557] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:01:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:01:42.216362 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c309b9f9-e5f0-4798-91ca-df2753209da3 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:42.218276 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-c309b9f9-e5f0-4798-91ca-df2753209da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:42.235243 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c309b9f9-e5f0-4798-91ca-df2753209da3 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:42.308449 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:01:42.322261 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea68e990-4997-41c2-83c2-34dc6ba46582 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:42.323354 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68e990-4997-41c2-83c2-34dc6ba46582 service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:42.323891 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68e990-4997-41c2-83c2-34dc6ba46582 service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:42.324391 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68e990-4997-41c2-83c2-34dc6ba46582 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:42.335692 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea68e990-4997-41c2-83c2-34dc6ba46582 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:42.338094 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea68e990-4997-41c2-83c2-34dc6ba46582 service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:42.350340 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1283/2558] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:01:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 23:01:42.545427 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a46da9f-9885-4de3-b89d-709407128990 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:42.547065 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-2a46da9f-9885-4de3-b89d-709407128990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:42.562936 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a46da9f-9885-4de3-b89d-709407128990 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:42.569261 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090006ce-8c1f-4953-8933-a53c353258df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:42.571867 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090006ce-8c1f-4953-8933-a53c353258df admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:42.574036 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090006ce-8c1f-4953-8933-a53c353258df admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:42.576825 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090006ce-8c1f-4953-8933-a53c353258df admin admin] PATH_INFO: `/v3/projects/8fe33afbf1594ff28ad628377258e4b8` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:42.578462 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090006ce-8c1f-4953-8933-a53c353258df admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fe33afbf1594ff28ad628377258e4b8)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:42.580678 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090006ce-8c1f-4953-8933-a53c353258df admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:42.712329 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-090006ce-8c1f-4953-8933-a53c353258df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fe33afbf1594ff28ad628377258e4b8', 'request_id': 'req-090006ce-8c1f-4953-8933-a53c353258df'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:42.718059 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-090006ce-8c1f-4953-8933-a53c353258df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fe33afbf1594ff28ad628377258e4b8', 'request_id': 'req-090006ce-8c1f-4953-8933-a53c353258df'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:42.803657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-090006ce-8c1f-4953-8933-a53c353258df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fe33afbf1594ff28ad628377258e4b8', 'request_id': 'req-090006ce-8c1f-4953-8933-a53c353258df'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:42.819359 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-090006ce-8c1f-4953-8933-a53c353258df admin admin] The token cache is being invalidate because project 8fe33afbf1594ff28ad628377258e4b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:01:42.819952 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-090006ce-8c1f-4953-8933-a53c353258df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:42.820669 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-090006ce-8c1f-4953-8933-a53c353258df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:42.822274 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1276/2559] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 23:01:42 2022] DELETE /identity/v3/projects/8fe33afbf1594ff28ad628377258e4b8 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:01:42.826730 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea68e990-4997-41c2-83c2-34dc6ba46582 service neutron] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:42.838016 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea68e990-4997-41c2-83c2-34dc6ba46582 service neutron] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:42.922983 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4453759-334b-428e-b33b-6c23dd5030cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:42.924374 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4453759-334b-428e-b33b-6c23dd5030cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:42.924747 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4453759-334b-428e-b33b-6c23dd5030cd admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:42.925039 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4453759-334b-428e-b33b-6c23dd5030cd admin admin] PATH_INFO: `/v3/users/cb5fac5daaa0428d8ae033a8b339c3fc` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:42.926379 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4453759-334b-428e-b33b-6c23dd5030cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb5fac5daaa0428d8ae033a8b339c3fc)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:42.928621 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4453759-334b-428e-b33b-6c23dd5030cd admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:43.210923 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-f4453759-334b-428e-b33b-6c23dd5030cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb5fac5daaa0428d8ae033a8b339c3fc'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:43.218823 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-f4453759-334b-428e-b33b-6c23dd5030cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb5fac5daaa0428d8ae033a8b339c3fc'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:43.219747 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-f4453759-334b-428e-b33b-6c23dd5030cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb5fac5daaa0428d8ae033a8b339c3fc'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:43.234142 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-f4453759-334b-428e-b33b-6c23dd5030cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb5fac5daaa0428d8ae033a8b339c3fc'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:43.326434 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-f4453759-334b-428e-b33b-6c23dd5030cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb5fac5daaa0428d8ae033a8b339c3fc'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:43.336848 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1284/2560] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 23:01:42 2022] DELETE /identity/v3/users/cb5fac5daaa0428d8ae033a8b339c3fc => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:01:43.870392 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090006ce-8c1f-4953-8933-a53c353258df admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:43.874083 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-090006ce-8c1f-4953-8933-a53c353258df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:43.902909 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090006ce-8c1f-4953-8933-a53c353258df admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:44.001435 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94158dab-f56f-428d-a4e1-a938a64364f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:44.004698 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94158dab-f56f-428d-a4e1-a938a64364f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:44.004698 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94158dab-f56f-428d-a4e1-a938a64364f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:44.004698 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94158dab-f56f-428d-a4e1-a938a64364f3 admin admin] PATH_INFO: `/v3/projects/fdf95f1643c348c9a497390026b83eef` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:44.005758 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94158dab-f56f-428d-a4e1-a938a64364f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdf95f1643c348c9a497390026b83eef)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:44.008016 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94158dab-f56f-428d-a4e1-a938a64364f3 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:44.176023 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-94158dab-f56f-428d-a4e1-a938a64364f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdf95f1643c348c9a497390026b83eef', 'request_id': 'req-94158dab-f56f-428d-a4e1-a938a64364f3'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:44.182174 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-94158dab-f56f-428d-a4e1-a938a64364f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdf95f1643c348c9a497390026b83eef', 'request_id': 'req-94158dab-f56f-428d-a4e1-a938a64364f3'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:44.283774 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-94158dab-f56f-428d-a4e1-a938a64364f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdf95f1643c348c9a497390026b83eef', 'request_id': 'req-94158dab-f56f-428d-a4e1-a938a64364f3'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:44.295916 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d103a0af-387f-40dd-acf0-948b3ae23d64 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:44.296195 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d103a0af-387f-40dd-acf0-948b3ae23d64 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:44.296656 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d103a0af-387f-40dd-acf0-948b3ae23d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:44.311166 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-94158dab-f56f-428d-a4e1-a938a64364f3 admin admin] The token cache is being invalidate because project fdf95f1643c348c9a497390026b83eef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:01:44.311536 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-94158dab-f56f-428d-a4e1-a938a64364f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:44.312861 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-94158dab-f56f-428d-a4e1-a938a64364f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:44.320635 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1277/2561] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 23:01:43 2022] DELETE /identity/v3/projects/fdf95f1643c348c9a497390026b83eef => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:01:44.322600 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16098851-d085-44ef-8a08-d52305097c4f None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:44.323125 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16098851-d085-44ef-8a08-d52305097c4f None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:44.323461 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16098851-d085-44ef-8a08-d52305097c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:44.331193 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:01:44.335752 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-d103a0af-387f-40dd-acf0-948b3ae23d64 None None] MFA Rules not processed for user `7146cf2476be44c6ab118e8737cdbb08`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:01:44.337192 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-d103a0af-387f-40dd-acf0-948b3ae23d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:44.347997 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-16098851-d085-44ef-8a08-d52305097c4f None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:01:44.349231 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-16098851-d085-44ef-8a08-d52305097c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:44.406649 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:01:44.423460 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1285/2562] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:01:44 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:01:44.435797 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d103a0af-387f-40dd-acf0-948b3ae23d64 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:44.437562 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-d103a0af-387f-40dd-acf0-948b3ae23d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:44.457690 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d103a0af-387f-40dd-acf0-948b3ae23d64 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:44.463400 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc64db06-9835-4ee8-b1a4-22df2a11d469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:44.464062 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64db06-9835-4ee8-b1a4-22df2a11d469 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:44.464342 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64db06-9835-4ee8-b1a4-22df2a11d469 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:44.464612 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64db06-9835-4ee8-b1a4-22df2a11d469 admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:44.465302 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc64db06-9835-4ee8-b1a4-22df2a11d469 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:44.465648 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1278/2563] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:01:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 23:01:44.467138 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e2f4c-05c9-4da1-8fa0-667f25204892 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:44.467462 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e2f4c-05c9-4da1-8fa0-667f25204892 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:44.468206 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e2f4c-05c9-4da1-8fa0-667f25204892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:44.469093 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc64db06-9835-4ee8-b1a4-22df2a11d469 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:44.497459 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-8d1e2f4c-05c9-4da1-8fa0-667f25204892 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:01:44.503188 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-8d1e2f4c-05c9-4da1-8fa0-667f25204892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:44.512493 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1279/2564] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:01:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:01:44.516629 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1e2f4c-05c9-4da1-8fa0-667f25204892 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:44.534200 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1e2f4c-05c9-4da1-8fa0-667f25204892 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:44.541009 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c0f611-c4cc-4de9-9e99-840869eb1b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:44.541702 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0f611-c4cc-4de9-9e99-840869eb1b58 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:44.541989 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0f611-c4cc-4de9-9e99-840869eb1b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:44.542269 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0f611-c4cc-4de9-9e99-840869eb1b58 admin admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:44.543039 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c0f611-c4cc-4de9-9e99-840869eb1b58 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 23:01:44.543917 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c0f611-c4cc-4de9-9e99-840869eb1b58 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:44.546579 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c0f611-c4cc-4de9-9e99-840869eb1b58 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:44.565744 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1280/2565] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 23:01:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 23:01:44.570222 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c0f611-c4cc-4de9-9e99-840869eb1b58 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:44.577342 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1286/2566] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 23:01:44 2022] POST /identity/v3/projects => generated 403 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:01:44.583939 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc64db06-9835-4ee8-b1a4-22df2a11d469 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:44.594947 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c0f611-c4cc-4de9-9e99-840869eb1b58 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:44.664727 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc64db06-9835-4ee8-b1a4-22df2a11d469 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:44.669416 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf109d-d9f8-4c8d-860d-fc3eacdd1737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:44.670094 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf109d-d9f8-4c8d-860d-fc3eacdd1737 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:44.670375 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf109d-d9f8-4c8d-860d-fc3eacdd1737 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:44.670657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf109d-d9f8-4c8d-860d-fc3eacdd1737 admin admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:44.671364 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbf109d-d9f8-4c8d-860d-fc3eacdd1737 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:44.672187 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a367c75-f446-4eca-a7b4-3534ae47586b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:44.672898 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a367c75-f446-4eca-a7b4-3534ae47586b admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:44.673184 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a367c75-f446-4eca-a7b4-3534ae47586b admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:44.673289 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbf109d-d9f8-4c8d-860d-fc3eacdd1737 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:44.673462 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a367c75-f446-4eca-a7b4-3534ae47586b admin admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:44.674146 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a367c75-f446-4eca-a7b4-3534ae47586b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:44.676231 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a367c75-f446-4eca-a7b4-3534ae47586b admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:44.810048 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1287/2567] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 23:01:44 2022] POST /identity/v3/users => generated 308 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:01:44.816475 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a367c75-f446-4eca-a7b4-3534ae47586b admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:44.822413 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a367c75-f446-4eca-a7b4-3534ae47586b admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:44.827177 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fccc13-4db5-4e86-abb0-4484e185eeb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:44.827885 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fccc13-4db5-4e86-abb0-4484e185eeb8 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:44.828204 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fccc13-4db5-4e86-abb0-4484e185eeb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:44.828485 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fccc13-4db5-4e86-abb0-4484e185eeb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:44.829218 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fccc13-4db5-4e86-abb0-4484e185eeb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:44.830838 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fccc13-4db5-4e86-abb0-4484e185eeb8 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:44.844811 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1288/2568] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:01:44.845996 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1281/2569] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 23:01:44 2022] POST /identity/v3/projects => generated 375 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 23:01:44.849277 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fccc13-4db5-4e86-abb0-4484e185eeb8 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:44.850369 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf109d-d9f8-4c8d-860d-fc3eacdd1737 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:44.860389 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fccc13-4db5-4e86-abb0-4484e185eeb8 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:44.919560 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3cc320-9462-4ed2-8be0-9077082abb66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:44.920489 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3cc320-9462-4ed2-8be0-9077082abb66 admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:44.920771 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3cc320-9462-4ed2-8be0-9077082abb66 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:44.921050 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3cc320-9462-4ed2-8be0-9077082abb66 admin admin] PATH_INFO: `/v3/projects/277967b705dd4bb5ba4567e755c4c788/users/ee2d0c58e04f4c5a9a0f29ab9a209583/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:44.927896 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf109d-d9f8-4c8d-860d-fc3eacdd1737 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:44.939366 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270249b2-a7bf-47b4-b9cb-f2a0ddf9e3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:44.940331 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270249b2-a7bf-47b4-b9cb-f2a0ddf9e3cf admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:44.940739 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270249b2-a7bf-47b4-b9cb-f2a0ddf9e3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:44.941146 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270249b2-a7bf-47b4-b9cb-f2a0ddf9e3cf admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:44.942039 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270249b2-a7bf-47b4-b9cb-f2a0ddf9e3cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:44.942217 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3cc320-9462-4ed2-8be0-9077082abb66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=277967b705dd4bb5ba4567e755c4c788, user_id=ee2d0c58e04f4c5a9a0f29ab9a209583, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:44.944768 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270249b2-a7bf-47b4-b9cb-f2a0ddf9e3cf admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:44.945571 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3cc320-9462-4ed2-8be0-9077082abb66 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:45.057814 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1289/2570] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 23:01:44 2022] PUT /identity/v3/projects/277967b705dd4bb5ba4567e755c4c788/users/ee2d0c58e04f4c5a9a0f29ab9a209583/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:01:45.063229 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3cc320-9462-4ed2-8be0-9077082abb66 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:45.090993 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1282/2571] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 23:01:44 2022] POST /identity/v3/users => generated 288 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 23:01:45.099076 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270249b2-a7bf-47b4-b9cb-f2a0ddf9e3cf admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:45.116429 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270249b2-a7bf-47b4-b9cb-f2a0ddf9e3cf admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:45.137073 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3cc320-9462-4ed2-8be0-9077082abb66 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:45.142318 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58d117c-6f46-4e91-a2ce-b72502aa8ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:45.143105 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d117c-6f46-4e91-a2ce-b72502aa8ae3 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:45.143449 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d117c-6f46-4e91-a2ce-b72502aa8ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:45.143793 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d117c-6f46-4e91-a2ce-b72502aa8ae3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:45.144694 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58d117c-6f46-4e91-a2ce-b72502aa8ae3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:45.146202 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58d117c-6f46-4e91-a2ce-b72502aa8ae3 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:45.158976 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1290/2572] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:01:45 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 23:01:45.164100 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58d117c-6f46-4e91-a2ce-b72502aa8ae3 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:45.173589 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58d117c-6f46-4e91-a2ce-b72502aa8ae3 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:45.178635 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780df5ea-fb0e-4824-9ea2-ed69702efba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:45.179790 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780df5ea-fb0e-4824-9ea2-ed69702efba0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:45.180082 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780df5ea-fb0e-4824-9ea2-ed69702efba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:45.180380 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780df5ea-fb0e-4824-9ea2-ed69702efba0 admin admin] PATH_INFO: `/v3/projects/277967b705dd4bb5ba4567e755c4c788/users/ee2d0c58e04f4c5a9a0f29ab9a209583/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:45.180870 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc13a3c-bc31-496f-bbdd-0a15eb93dafa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:45.181568 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc13a3c-bc31-496f-bbdd-0a15eb93dafa admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:45.181997 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc13a3c-bc31-496f-bbdd-0a15eb93dafa admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:45.182236 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc13a3c-bc31-496f-bbdd-0a15eb93dafa admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:45.182996 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc13a3c-bc31-496f-bbdd-0a15eb93dafa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:45.184759 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780df5ea-fb0e-4824-9ea2-ed69702efba0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=277967b705dd4bb5ba4567e755c4c788, user_id=ee2d0c58e04f4c5a9a0f29ab9a209583, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:45.185215 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc13a3c-bc31-496f-bbdd-0a15eb93dafa admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:45.187518 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780df5ea-fb0e-4824-9ea2-ed69702efba0 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:45.196856 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1283/2573] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:01:45 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:01:45.198585 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1291/2574] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 23:01:45 2022] PUT /identity/v3/projects/277967b705dd4bb5ba4567e755c4c788/users/ee2d0c58e04f4c5a9a0f29ab9a209583/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 23:01:45.201317 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc13a3c-bc31-496f-bbdd-0a15eb93dafa admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:45.216480 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc13a3c-bc31-496f-bbdd-0a15eb93dafa admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:45.276871 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a1756c-1649-438b-ba90-a987ea207dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:45.277646 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a1756c-1649-438b-ba90-a987ea207dcb admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:45.277930 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a1756c-1649-438b-ba90-a987ea207dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:45.278204 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a1756c-1649-438b-ba90-a987ea207dcb admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:45.279128 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a1756c-1649-438b-ba90-a987ea207dcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:45.281077 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a1756c-1649-438b-ba90-a987ea207dcb admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:45.292007 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1284/2575] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:01:45 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 23:01:45.296279 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780df5ea-fb0e-4824-9ea2-ed69702efba0 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:45.307301 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780df5ea-fb0e-4824-9ea2-ed69702efba0 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:45.312213 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c122dc6c-d213-46dd-9138-cfa74fe9ba56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:45.313134 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c122dc6c-d213-46dd-9138-cfa74fe9ba56 admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:45.313411 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c122dc6c-d213-46dd-9138-cfa74fe9ba56 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:45.313684 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c122dc6c-d213-46dd-9138-cfa74fe9ba56 admin admin] PATH_INFO: `/v3/projects/fe53d6ca4fa44a67ae4a039635d2c5b6/users/3439a666c26740f9b27d78cd721f5fa6/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:45.329749 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c122dc6c-d213-46dd-9138-cfa74fe9ba56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe53d6ca4fa44a67ae4a039635d2c5b6, user_id=3439a666c26740f9b27d78cd721f5fa6, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:45.332449 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c122dc6c-d213-46dd-9138-cfa74fe9ba56 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:45.471350 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1292/2576] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 23:01:45 2022] PUT /identity/v3/projects/fe53d6ca4fa44a67ae4a039635d2c5b6/users/3439a666c26740f9b27d78cd721f5fa6/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 23:01:45.484944 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731260b0-3156-4f4a-8539-556fb6d6be5a None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:45.485243 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731260b0-3156-4f4a-8539-556fb6d6be5a None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:45.485563 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731260b0-3156-4f4a-8539-556fb6d6be5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:45.520918 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-731260b0-3156-4f4a-8539-556fb6d6be5a None None] MFA Rules not processed for user `3439a666c26740f9b27d78cd721f5fa6`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:01:45.523442 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-731260b0-3156-4f4a-8539-556fb6d6be5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:45.622040 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1285/2577] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:01:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:01:45.637697 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c122dc6c-d213-46dd-9138-cfa74fe9ba56 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:45.639940 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-c122dc6c-d213-46dd-9138-cfa74fe9ba56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:45.725487 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c122dc6c-d213-46dd-9138-cfa74fe9ba56 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:45.732842 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72907fcd-4873-47c5-acfe-2920ba292bf4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:45.733733 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72907fcd-4873-47c5-acfe-2920ba292bf4 service cinder] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:45.734023 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72907fcd-4873-47c5-acfe-2920ba292bf4 service cinder] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:45.734300 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72907fcd-4873-47c5-acfe-2920ba292bf4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:45.745097 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72907fcd-4873-47c5-acfe-2920ba292bf4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:45.746526 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72907fcd-4873-47c5-acfe-2920ba292bf4 service cinder] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:45.757538 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1293/2578] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:01:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:01:46.458476 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:01:46.635325 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:01:46.741009 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f959f5-8b63-478e-8df0-85d76e677d78 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:46.741299 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f959f5-8b63-478e-8df0-85d76e677d78 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:46.741577 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f959f5-8b63-478e-8df0-85d76e677d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:46.772494 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-a1f959f5-8b63-478e-8df0-85d76e677d78 None None] MFA Rules not processed for user `ee2d0c58e04f4c5a9a0f29ab9a209583`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:01:46.773830 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-a1f959f5-8b63-478e-8df0-85d76e677d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:46.845298 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1286/2579] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:01:46 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:01:47.612726 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=100 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:01:48.744608 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72907fcd-4873-47c5-acfe-2920ba292bf4 service cinder] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:48.839750 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72907fcd-4873-47c5-acfe-2920ba292bf4 service cinder] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:48.850275 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b84187d-3519-41c7-9b9b-b55d23af50d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:48.850275 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84187d-3519-41c7-9b9b-b55d23af50d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:48.850275 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84187d-3519-41c7-9b9b-b55d23af50d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:48.850275 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84187d-3519-41c7-9b9b-b55d23af50d8 admin admin] PATH_INFO: `/v3/users/7146cf2476be44c6ab118e8737cdbb08` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:48.850703 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b84187d-3519-41c7-9b9b-b55d23af50d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7146cf2476be44c6ab118e8737cdbb08)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:48.852591 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b84187d-3519-41c7-9b9b-b55d23af50d8 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:49.045772 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:01:49.055050 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-3b84187d-3519-41c7-9b9b-b55d23af50d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7146cf2476be44c6ab118e8737cdbb08'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:49.063213 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-3b84187d-3519-41c7-9b9b-b55d23af50d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7146cf2476be44c6ab118e8737cdbb08'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:49.064025 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-3b84187d-3519-41c7-9b9b-b55d23af50d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7146cf2476be44c6ab118e8737cdbb08'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:49.077820 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-3b84187d-3519-41c7-9b9b-b55d23af50d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7146cf2476be44c6ab118e8737cdbb08'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:49.137157 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-3b84187d-3519-41c7-9b9b-b55d23af50d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7146cf2476be44c6ab118e8737cdbb08'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:49.144477 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1294/2580] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 23:01:48 2022] DELETE /identity/v3/users/7146cf2476be44c6ab118e8737cdbb08 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:01:49.519733 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f959f5-8b63-478e-8df0-85d76e677d78 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:49.533472 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f959f5-8b63-478e-8df0-85d76e677d78 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:49.607347 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f03ab-7a6a-4ff9-8b05-d595e55cd5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:49.608297 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f03ab-7a6a-4ff9-8b05-d595e55cd5f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:49.608583 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f03ab-7a6a-4ff9-8b05-d595e55cd5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:49.608861 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f03ab-7a6a-4ff9-8b05-d595e55cd5f5 admin admin] PATH_INFO: `/v3/projects/fe4174e900e542c992af44c90f4e7fd2` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:49.610071 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76f03ab-7a6a-4ff9-8b05-d595e55cd5f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe4174e900e542c992af44c90f4e7fd2)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:49.611998 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76f03ab-7a6a-4ff9-8b05-d595e55cd5f5 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:49.722955 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-c76f03ab-7a6a-4ff9-8b05-d595e55cd5f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe4174e900e542c992af44c90f4e7fd2', 'request_id': 'req-c76f03ab-7a6a-4ff9-8b05-d595e55cd5f5'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:49.728010 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-c76f03ab-7a6a-4ff9-8b05-d595e55cd5f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe4174e900e542c992af44c90f4e7fd2', 'request_id': 'req-c76f03ab-7a6a-4ff9-8b05-d595e55cd5f5'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:49.804357 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-c76f03ab-7a6a-4ff9-8b05-d595e55cd5f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe4174e900e542c992af44c90f4e7fd2', 'request_id': 'req-c76f03ab-7a6a-4ff9-8b05-d595e55cd5f5'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:49.816738 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-c76f03ab-7a6a-4ff9-8b05-d595e55cd5f5 admin admin] The token cache is being invalidate because project fe4174e900e542c992af44c90f4e7fd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:01:49.817025 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-c76f03ab-7a6a-4ff9-8b05-d595e55cd5f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:49.817661 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-c76f03ab-7a6a-4ff9-8b05-d595e55cd5f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:49.819919 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1287/2581] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 23:01:49 2022] DELETE /identity/v3/projects/fe4174e900e542c992af44c90f4e7fd2 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:01:49.823919 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b84187d-3519-41c7-9b9b-b55d23af50d8 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:49.825412 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-3b84187d-3519-41c7-9b9b-b55d23af50d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:49.838426 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b84187d-3519-41c7-9b9b-b55d23af50d8 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:49.897881 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc38b268-accb-465b-a8f0-005c79bd5f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:49.898767 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc38b268-accb-465b-a8f0-005c79bd5f29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:49.899075 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc38b268-accb-465b-a8f0-005c79bd5f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:49.899354 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc38b268-accb-465b-a8f0-005c79bd5f29 admin admin] PATH_INFO: `/v3/users/ee2d0c58e04f4c5a9a0f29ab9a209583` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:49.900587 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc38b268-accb-465b-a8f0-005c79bd5f29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee2d0c58e04f4c5a9a0f29ab9a209583)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:49.903589 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc38b268-accb-465b-a8f0-005c79bd5f29 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:50.140185 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-bc38b268-accb-465b-a8f0-005c79bd5f29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee2d0c58e04f4c5a9a0f29ab9a209583'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:50.147735 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-bc38b268-accb-465b-a8f0-005c79bd5f29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee2d0c58e04f4c5a9a0f29ab9a209583'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:50.148401 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-bc38b268-accb-465b-a8f0-005c79bd5f29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee2d0c58e04f4c5a9a0f29ab9a209583'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:50.161231 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-bc38b268-accb-465b-a8f0-005c79bd5f29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee2d0c58e04f4c5a9a0f29ab9a209583'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:50.262887 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-bc38b268-accb-465b-a8f0-005c79bd5f29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee2d0c58e04f4c5a9a0f29ab9a209583'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:50.271238 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1295/2582] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 23:01:49 2022] DELETE /identity/v3/users/ee2d0c58e04f4c5a9a0f29ab9a209583 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:01:50.641517 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f03ab-7a6a-4ff9-8b05-d595e55cd5f5 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:50.643090 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-c76f03ab-7a6a-4ff9-8b05-d595e55cd5f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:50.656694 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f03ab-7a6a-4ff9-8b05-d595e55cd5f5 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:50.690315 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:01:50.715018 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c4444-f3bc-48a8-a8e0-3d089fd5ea2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:50.715983 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c4444-f3bc-48a8-a8e0-3d089fd5ea2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:50.716150 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c4444-f3bc-48a8-a8e0-3d089fd5ea2b admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:50.716425 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c4444-f3bc-48a8-a8e0-3d089fd5ea2b admin admin] PATH_INFO: `/v3/projects/277967b705dd4bb5ba4567e755c4c788` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:50.717609 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14c4444-f3bc-48a8-a8e0-3d089fd5ea2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=277967b705dd4bb5ba4567e755c4c788)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:50.719562 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14c4444-f3bc-48a8-a8e0-3d089fd5ea2b admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:50.830123 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-d14c4444-f3bc-48a8-a8e0-3d089fd5ea2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '277967b705dd4bb5ba4567e755c4c788', 'request_id': 'req-d14c4444-f3bc-48a8-a8e0-3d089fd5ea2b'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:50.836232 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-d14c4444-f3bc-48a8-a8e0-3d089fd5ea2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '277967b705dd4bb5ba4567e755c4c788', 'request_id': 'req-d14c4444-f3bc-48a8-a8e0-3d089fd5ea2b'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:50.891413 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-d14c4444-f3bc-48a8-a8e0-3d089fd5ea2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '277967b705dd4bb5ba4567e755c4c788', 'request_id': 'req-d14c4444-f3bc-48a8-a8e0-3d089fd5ea2b'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:50.903064 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-d14c4444-f3bc-48a8-a8e0-3d089fd5ea2b admin admin] The token cache is being invalidate because project 277967b705dd4bb5ba4567e755c4c788 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:01:50.903828 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-d14c4444-f3bc-48a8-a8e0-3d089fd5ea2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:50.903943 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-d14c4444-f3bc-48a8-a8e0-3d089fd5ea2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:50.905793 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1288/2583] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 23:01:50 2022] DELETE /identity/v3/projects/277967b705dd4bb5ba4567e755c4c788 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:01:50.910459 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afecd016-03ae-4a50-b007-3e4966039fcb None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:50.910980 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afecd016-03ae-4a50-b007-3e4966039fcb None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:50.911088 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afecd016-03ae-4a50-b007-3e4966039fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:50.928744 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-afecd016-03ae-4a50-b007-3e4966039fcb None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:01:50.929950 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-afecd016-03ae-4a50-b007-3e4966039fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:50.960093 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:01:50.993688 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1296/2584] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:01:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:01:51.001381 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165c5fd5-db88-4086-b598-0a242001e93c None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:51.001669 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165c5fd5-db88-4086-b598-0a242001e93c None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:51.001947 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165c5fd5-db88-4086-b598-0a242001e93c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:51.020713 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-165c5fd5-db88-4086-b598-0a242001e93c None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:01:51.024157 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-165c5fd5-db88-4086-b598-0a242001e93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:51.029662 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1289/2585] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:01:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:01:51.033779 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afecd016-03ae-4a50-b007-3e4966039fcb None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:51.043444 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afecd016-03ae-4a50-b007-3e4966039fcb None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:51.048057 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cc4abc-e8e1-46a0-9007-1ee41c43bd2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:51.048974 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc4abc-e8e1-46a0-9007-1ee41c43bd2c admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:51.049425 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc4abc-e8e1-46a0-9007-1ee41c43bd2c admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:51.049998 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc4abc-e8e1-46a0-9007-1ee41c43bd2c admin admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:51.050755 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cc4abc-e8e1-46a0-9007-1ee41c43bd2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 23:01:51.051145 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cc4abc-e8e1-46a0-9007-1ee41c43bd2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:51.053282 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cc4abc-e8e1-46a0-9007-1ee41c43bd2c admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:51.064634 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1297/2586] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 23:01:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 23:01:51.069286 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165c5fd5-db88-4086-b598-0a242001e93c None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:51.078345 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165c5fd5-db88-4086-b598-0a242001e93c None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:51.082886 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fac05d-3e54-4e15-aac7-0080e6ecbfbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:51.083522 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fac05d-3e54-4e15-aac7-0080e6ecbfbf admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:51.083804 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fac05d-3e54-4e15-aac7-0080e6ecbfbf admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:51.084148 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fac05d-3e54-4e15-aac7-0080e6ecbfbf admin admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:51.084828 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fac05d-3e54-4e15-aac7-0080e6ecbfbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:51.086694 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fac05d-3e54-4e15-aac7-0080e6ecbfbf admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:51.194255 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1290/2587] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 23:01:51 2022] POST /identity/v3/projects => generated 387 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:01:51.198404 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cc4abc-e8e1-46a0-9007-1ee41c43bd2c admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:51.208399 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cc4abc-e8e1-46a0-9007-1ee41c43bd2c admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:51.264434 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8067506b-5f79-43dd-b89c-7823784b2f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:51.265100 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8067506b-5f79-43dd-b89c-7823784b2f73 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:51.265385 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8067506b-5f79-43dd-b89c-7823784b2f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:51.265665 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8067506b-5f79-43dd-b89c-7823784b2f73 admin admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:51.266342 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8067506b-5f79-43dd-b89c-7823784b2f73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:51.268280 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8067506b-5f79-43dd-b89c-7823784b2f73 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:51.341283 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1298/2588] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 23:01:51 2022] POST /identity/v3/users => generated 294 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:01:51.345315 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fac05d-3e54-4e15-aac7-0080e6ecbfbf admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:51.356778 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fac05d-3e54-4e15-aac7-0080e6ecbfbf admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:51.361423 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4a213f-97b5-4088-ac3f-908e85b3e01e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:51.362110 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4a213f-97b5-4088-ac3f-908e85b3e01e admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:51.362426 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4a213f-97b5-4088-ac3f-908e85b3e01e admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:51.362705 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4a213f-97b5-4088-ac3f-908e85b3e01e admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:51.363498 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4a213f-97b5-4088-ac3f-908e85b3e01e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:51.365090 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4a213f-97b5-4088-ac3f-908e85b3e01e admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:51.375726 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1291/2589] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:01:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:01:51.379566 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8067506b-5f79-43dd-b89c-7823784b2f73 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:51.389934 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8067506b-5f79-43dd-b89c-7823784b2f73 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:51.394593 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef8f5ee-3a6c-4ef1-b80c-cc8827bf6134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:51.395627 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef8f5ee-3a6c-4ef1-b80c-cc8827bf6134 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:51.395906 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef8f5ee-3a6c-4ef1-b80c-cc8827bf6134 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:51.396199 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef8f5ee-3a6c-4ef1-b80c-cc8827bf6134 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:51.397638 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef8f5ee-3a6c-4ef1-b80c-cc8827bf6134 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:51.399238 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef8f5ee-3a6c-4ef1-b80c-cc8827bf6134 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:51.411731 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1299/2590] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:01:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:01:51.413559 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4a213f-97b5-4088-ac3f-908e85b3e01e admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:51.424011 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4a213f-97b5-4088-ac3f-908e85b3e01e admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:51.429509 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb584a1-3e3d-4168-9574-940eeb98b0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:51.430209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb584a1-3e3d-4168-9574-940eeb98b0e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:51.430612 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb584a1-3e3d-4168-9574-940eeb98b0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:51.430996 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb584a1-3e3d-4168-9574-940eeb98b0e6 admin admin] PATH_INFO: `/v3/projects/edfb8e6ea7b548b792003568fa26fa93/users/cd33ed7b85b4468e84c3a12aa33f8c03/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:51.448950 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb584a1-3e3d-4168-9574-940eeb98b0e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edfb8e6ea7b548b792003568fa26fa93, user_id=cd33ed7b85b4468e84c3a12aa33f8c03, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:51.449511 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb584a1-3e3d-4168-9574-940eeb98b0e6 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:51.515141 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1292/2591] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 23:01:51 2022] PUT /identity/v3/projects/edfb8e6ea7b548b792003568fa26fa93/users/cd33ed7b85b4468e84c3a12aa33f8c03/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:01:51.526091 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43892c-fb7d-4fc1-937c-a2b4a24abbc0 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:51.526379 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43892c-fb7d-4fc1-937c-a2b4a24abbc0 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:51.526697 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43892c-fb7d-4fc1-937c-a2b4a24abbc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:51.552591 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-fd43892c-fb7d-4fc1-937c-a2b4a24abbc0 None None] MFA Rules not processed for user `cd33ed7b85b4468e84c3a12aa33f8c03`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:01:51.553861 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-fd43892c-fb7d-4fc1-937c-a2b4a24abbc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:51.615699 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1300/2592] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:01:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:01:51.620206 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb584a1-3e3d-4168-9574-940eeb98b0e6 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:51.698467 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb584a1-3e3d-4168-9574-940eeb98b0e6 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:51.704049 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d467c2-7c00-43e7-9d9b-44717a0cadb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:51.704712 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d467c2-7c00-43e7-9d9b-44717a0cadb1 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:51.704993 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d467c2-7c00-43e7-9d9b-44717a0cadb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:51.705269 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d467c2-7c00-43e7-9d9b-44717a0cadb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:51.706083 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d467c2-7c00-43e7-9d9b-44717a0cadb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:51.708035 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d467c2-7c00-43e7-9d9b-44717a0cadb1 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:51.714540 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:01:51.817382 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1293/2593] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 23:01:51 2022] POST /identity/v3/projects => generated 387 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:01:51.823472 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd43892c-fb7d-4fc1-937c-a2b4a24abbc0 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:51.835185 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd43892c-fb7d-4fc1-937c-a2b4a24abbc0 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:51.865113 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:01:51.907649 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2776711-cc65-44e6-90fd-9ffd4801aa77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:51.909015 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2776711-cc65-44e6-90fd-9ffd4801aa77 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:51.909317 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2776711-cc65-44e6-90fd-9ffd4801aa77 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:51.909606 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2776711-cc65-44e6-90fd-9ffd4801aa77 admin admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:51.911320 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2776711-cc65-44e6-90fd-9ffd4801aa77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:51.913351 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2776711-cc65-44e6-90fd-9ffd4801aa77 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:52.001164 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1301/2594] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 23:01:51 2022] POST /identity/v3/users => generated 300 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:01:52.005275 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d467c2-7c00-43e7-9d9b-44717a0cadb1 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:52.016599 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d467c2-7c00-43e7-9d9b-44717a0cadb1 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:52.021559 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8572fe30-3f23-4ca5-8c1f-6e6e4c292049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:52.022301 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8572fe30-3f23-4ca5-8c1f-6e6e4c292049 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:52.022660 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8572fe30-3f23-4ca5-8c1f-6e6e4c292049 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:52.023054 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8572fe30-3f23-4ca5-8c1f-6e6e4c292049 admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:52.023851 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8572fe30-3f23-4ca5-8c1f-6e6e4c292049 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:52.025524 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8572fe30-3f23-4ca5-8c1f-6e6e4c292049 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:52.036603 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1294/2595] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:01:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:01:52.042260 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2776711-cc65-44e6-90fd-9ffd4801aa77 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:52.054407 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2776711-cc65-44e6-90fd-9ffd4801aa77 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:52.059448 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b6e792-869b-403e-9450-6948db204fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:52.060337 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b6e792-869b-403e-9450-6948db204fa0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:52.060638 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b6e792-869b-403e-9450-6948db204fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:52.060950 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b6e792-869b-403e-9450-6948db204fa0 admin admin] PATH_INFO: `/v3/projects/f620736d59864b178d7210f116b2be48/users/3f3144e547c14636a1bd8db309b92546/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:52.077647 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b6e792-869b-403e-9450-6948db204fa0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f620736d59864b178d7210f116b2be48, user_id=3f3144e547c14636a1bd8db309b92546, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:52.080397 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b6e792-869b-403e-9450-6948db204fa0 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:52.172594 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1302/2596] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 23:01:52 2022] PUT /identity/v3/projects/f620736d59864b178d7210f116b2be48/users/3f3144e547c14636a1bd8db309b92546/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:01:52.176790 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8572fe30-3f23-4ca5-8c1f-6e6e4c292049 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:52.188293 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8572fe30-3f23-4ca5-8c1f-6e6e4c292049 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:52.256763 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55992e7-fff1-44f9-bc70-453781800a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:52.257617 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55992e7-fff1-44f9-bc70-453781800a49 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:52.258006 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55992e7-fff1-44f9-bc70-453781800a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:52.258386 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55992e7-fff1-44f9-bc70-453781800a49 admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:52.259911 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55992e7-fff1-44f9-bc70-453781800a49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:52.261872 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55992e7-fff1-44f9-bc70-453781800a49 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:52.275149 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1295/2597] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:01:52 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:01:52.278976 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b6e792-869b-403e-9450-6948db204fa0 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:52.295447 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b6e792-869b-403e-9450-6948db204fa0 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:52.304014 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6527ec-80e6-4e93-9e6d-78d0f2ab44fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:52.305837 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6527ec-80e6-4e93-9e6d-78d0f2ab44fc admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:52.305837 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6527ec-80e6-4e93-9e6d-78d0f2ab44fc admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:52.305837 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6527ec-80e6-4e93-9e6d-78d0f2ab44fc admin admin] PATH_INFO: `/v3/projects/f620736d59864b178d7210f116b2be48/users/3f3144e547c14636a1bd8db309b92546/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:52.308112 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6527ec-80e6-4e93-9e6d-78d0f2ab44fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f620736d59864b178d7210f116b2be48, user_id=3f3144e547c14636a1bd8db309b92546, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:52.312314 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6527ec-80e6-4e93-9e6d-78d0f2ab44fc admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:52.326147 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1303/2598] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 23:01:52 2022] PUT /identity/v3/projects/f620736d59864b178d7210f116b2be48/users/3f3144e547c14636a1bd8db309b92546/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 23:01:52.337818 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c87a7-c63a-445f-86e1-c0a1e40721d4 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:52.338118 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c87a7-c63a-445f-86e1-c0a1e40721d4 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:52.338405 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c87a7-c63a-445f-86e1-c0a1e40721d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:52.371699 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-106c87a7-c63a-445f-86e1-c0a1e40721d4 None None] MFA Rules not processed for user `3f3144e547c14636a1bd8db309b92546`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:01:52.373155 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-106c87a7-c63a-445f-86e1-c0a1e40721d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:52.458574 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1296/2599] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:01:52 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:01:52.514380 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6527ec-80e6-4e93-9e6d-78d0f2ab44fc admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:52.516163 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-6d6527ec-80e6-4e93-9e6d-78d0f2ab44fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:52.597322 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6527ec-80e6-4e93-9e6d-78d0f2ab44fc admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:52.605001 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89436655-105f-4e16-bafb-e3d610e98aa2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:52.606473 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89436655-105f-4e16-bafb-e3d610e98aa2 service cinder] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:52.606877 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89436655-105f-4e16-bafb-e3d610e98aa2 service cinder] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:52.607603 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89436655-105f-4e16-bafb-e3d610e98aa2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:52.634532 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89436655-105f-4e16-bafb-e3d610e98aa2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:52.636158 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89436655-105f-4e16-bafb-e3d610e98aa2 service cinder] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:52.651258 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1304/2600] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:01:52 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:01:52.739191 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106c87a7-c63a-445f-86e1-c0a1e40721d4 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:52.812408 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:01:52.817406 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106c87a7-c63a-445f-86e1-c0a1e40721d4 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:52.823025 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3384772f-ac03-48cc-b02b-1fb4abf629ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:52.823943 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384772f-ac03-48cc-b02b-1fb4abf629ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:52.824238 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384772f-ac03-48cc-b02b-1fb4abf629ca admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:52.824525 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384772f-ac03-48cc-b02b-1fb4abf629ca admin admin] PATH_INFO: `/v3/users/cd33ed7b85b4468e84c3a12aa33f8c03` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:52.825996 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3384772f-ac03-48cc-b02b-1fb4abf629ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd33ed7b85b4468e84c3a12aa33f8c03)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:52.828738 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3384772f-ac03-48cc-b02b-1fb4abf629ca admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:53.043315 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-3384772f-ac03-48cc-b02b-1fb4abf629ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd33ed7b85b4468e84c3a12aa33f8c03'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:53.051660 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-3384772f-ac03-48cc-b02b-1fb4abf629ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd33ed7b85b4468e84c3a12aa33f8c03'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:53.052602 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-3384772f-ac03-48cc-b02b-1fb4abf629ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd33ed7b85b4468e84c3a12aa33f8c03'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:53.069797 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-3384772f-ac03-48cc-b02b-1fb4abf629ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd33ed7b85b4468e84c3a12aa33f8c03'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:53.070529 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-3384772f-ac03-48cc-b02b-1fb4abf629ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd33ed7b85b4468e84c3a12aa33f8c03'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:53.154319 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1297/2601] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 23:01:52 2022] DELETE /identity/v3/users/cd33ed7b85b4468e84c3a12aa33f8c03 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 23:01:53.159033 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d29de3-1284-4895-af54-0322a61fbcdc None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:53.159851 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d29de3-1284-4895-af54-0322a61fbcdc None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:53.160483 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d29de3-1284-4895-af54-0322a61fbcdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:53.183513 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-81d29de3-1284-4895-af54-0322a61fbcdc None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:01:53.185182 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-81d29de3-1284-4895-af54-0322a61fbcdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:53.263051 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1305/2602] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:01:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:01:53.273159 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3384772f-ac03-48cc-b02b-1fb4abf629ca admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:53.274750 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-3384772f-ac03-48cc-b02b-1fb4abf629ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:53.342931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3384772f-ac03-48cc-b02b-1fb4abf629ca admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:53.343293 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2165d4-cd45-4ca0-b5e9-c9a6586e490e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:53.343847 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2165d4-cd45-4ca0-b5e9-c9a6586e490e service neutron] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:53.344184 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2165d4-cd45-4ca0-b5e9-c9a6586e490e service neutron] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:53.344459 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2165d4-cd45-4ca0-b5e9-c9a6586e490e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:53.356189 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2165d4-cd45-4ca0-b5e9-c9a6586e490e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:53.357663 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2165d4-cd45-4ca0-b5e9-c9a6586e490e service neutron] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:53.368306 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1298/2603] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:01:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:01:53.786099 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d29de3-1284-4895-af54-0322a61fbcdc None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:53.788027 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-81d29de3-1284-4895-af54-0322a61fbcdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:53.801554 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d29de3-1284-4895-af54-0322a61fbcdc None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:53.807933 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3a6412-f3b9-4d46-924c-4eb83fcdcc2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:53.808900 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3a6412-f3b9-4d46-924c-4eb83fcdcc2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:53.809283 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3a6412-f3b9-4d46-924c-4eb83fcdcc2e admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:53.809655 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3a6412-f3b9-4d46-924c-4eb83fcdcc2e admin admin] PATH_INFO: `/v3/projects/edfb8e6ea7b548b792003568fa26fa93` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:53.811241 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3a6412-f3b9-4d46-924c-4eb83fcdcc2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=edfb8e6ea7b548b792003568fa26fa93)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:53.813197 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3a6412-f3b9-4d46-924c-4eb83fcdcc2e admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:53.839206 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:01:53.956670 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-0a3a6412-f3b9-4d46-924c-4eb83fcdcc2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edfb8e6ea7b548b792003568fa26fa93', 'request_id': 'req-0a3a6412-f3b9-4d46-924c-4eb83fcdcc2e'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:53.962942 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-0a3a6412-f3b9-4d46-924c-4eb83fcdcc2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edfb8e6ea7b548b792003568fa26fa93', 'request_id': 'req-0a3a6412-f3b9-4d46-924c-4eb83fcdcc2e'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:54.037981 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-0a3a6412-f3b9-4d46-924c-4eb83fcdcc2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edfb8e6ea7b548b792003568fa26fa93', 'request_id': 'req-0a3a6412-f3b9-4d46-924c-4eb83fcdcc2e'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:54.049970 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-0a3a6412-f3b9-4d46-924c-4eb83fcdcc2e admin admin] The token cache is being invalidate because project edfb8e6ea7b548b792003568fa26fa93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:01:54.050260 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-0a3a6412-f3b9-4d46-924c-4eb83fcdcc2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:54.050945 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-0a3a6412-f3b9-4d46-924c-4eb83fcdcc2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:54.055846 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1306/2604] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 23:01:53 2022] DELETE /identity/v3/projects/edfb8e6ea7b548b792003568fa26fa93 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:01:54.058710 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2165d4-cd45-4ca0-b5e9-c9a6586e490e service neutron] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:54.069359 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2165d4-cd45-4ca0-b5e9-c9a6586e490e service neutron] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:54.132499 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48471c6c-2e60-4e92-935a-14e56936515d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:54.135921 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48471c6c-2e60-4e92-935a-14e56936515d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:54.135921 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48471c6c-2e60-4e92-935a-14e56936515d admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:54.135921 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48471c6c-2e60-4e92-935a-14e56936515d admin admin] PATH_INFO: `/v3/users/3f3144e547c14636a1bd8db309b92546` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:54.135921 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48471c6c-2e60-4e92-935a-14e56936515d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f3144e547c14636a1bd8db309b92546)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:54.137780 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48471c6c-2e60-4e92-935a-14e56936515d admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:54.171376 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:01:54.558292 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-48471c6c-2e60-4e92-935a-14e56936515d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f3144e547c14636a1bd8db309b92546'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:54.565414 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-48471c6c-2e60-4e92-935a-14e56936515d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f3144e547c14636a1bd8db309b92546'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:54.566089 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-48471c6c-2e60-4e92-935a-14e56936515d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f3144e547c14636a1bd8db309b92546'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:54.579701 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-48471c6c-2e60-4e92-935a-14e56936515d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f3144e547c14636a1bd8db309b92546'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:54.580403 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-48471c6c-2e60-4e92-935a-14e56936515d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f3144e547c14636a1bd8db309b92546'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:54.685979 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1299/2605] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 23:01:54 2022] DELETE /identity/v3/users/3f3144e547c14636a1bd8db309b92546 => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:01:55.062045 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3a6412-f3b9-4d46-924c-4eb83fcdcc2e admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:55.063654 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-0a3a6412-f3b9-4d46-924c-4eb83fcdcc2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:55.077127 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3a6412-f3b9-4d46-924c-4eb83fcdcc2e admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:55.139594 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9192d47-d835-4701-b958-3a38c5c4071f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:55.140454 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9192d47-d835-4701-b958-3a38c5c4071f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:55.140738 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9192d47-d835-4701-b958-3a38c5c4071f admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:55.141011 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9192d47-d835-4701-b958-3a38c5c4071f admin admin] PATH_INFO: `/v3/projects/f620736d59864b178d7210f116b2be48` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:55.142172 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9192d47-d835-4701-b958-3a38c5c4071f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f620736d59864b178d7210f116b2be48)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:55.144070 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9192d47-d835-4701-b958-3a38c5c4071f admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:55.278570 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-c9192d47-d835-4701-b958-3a38c5c4071f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f620736d59864b178d7210f116b2be48', 'request_id': 'req-c9192d47-d835-4701-b958-3a38c5c4071f'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:55.284481 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-c9192d47-d835-4701-b958-3a38c5c4071f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f620736d59864b178d7210f116b2be48', 'request_id': 'req-c9192d47-d835-4701-b958-3a38c5c4071f'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:55.358052 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-c9192d47-d835-4701-b958-3a38c5c4071f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f620736d59864b178d7210f116b2be48', 'request_id': 'req-c9192d47-d835-4701-b958-3a38c5c4071f'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:55.370554 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-c9192d47-d835-4701-b958-3a38c5c4071f admin admin] The token cache is being invalidate because project f620736d59864b178d7210f116b2be48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:01:55.370871 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-c9192d47-d835-4701-b958-3a38c5c4071f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:55.371530 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-c9192d47-d835-4701-b958-3a38c5c4071f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:55.373060 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1307/2606] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 23:01:55 2022] DELETE /identity/v3/projects/f620736d59864b178d7210f116b2be48 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:01:55.377989 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3982cb23-add7-4771-8a90-2c792e8d0c79 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:55.378536 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3982cb23-add7-4771-8a90-2c792e8d0c79 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:55.378643 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3982cb23-add7-4771-8a90-2c792e8d0c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:55.398514 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-3982cb23-add7-4771-8a90-2c792e8d0c79 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:01:55.399876 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-3982cb23-add7-4771-8a90-2c792e8d0c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:55.462341 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1300/2607] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:01:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:01:55.469951 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f41d7b0-2303-4852-8e42-267034f8d62f None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:55.470265 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f41d7b0-2303-4852-8e42-267034f8d62f None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:55.470563 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f41d7b0-2303-4852-8e42-267034f8d62f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:55.489256 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-8f41d7b0-2303-4852-8e42-267034f8d62f None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:01:55.490998 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-8f41d7b0-2303-4852-8e42-267034f8d62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:55.499346 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1308/2608] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:01:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:01:55.503344 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3982cb23-add7-4771-8a90-2c792e8d0c79 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:55.513964 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3982cb23-add7-4771-8a90-2c792e8d0c79 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:55.518692 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00da9f34-cb16-4fce-ae88-98611ca11280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:55.519424 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da9f34-cb16-4fce-ae88-98611ca11280 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:55.519715 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da9f34-cb16-4fce-ae88-98611ca11280 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:55.519998 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da9f34-cb16-4fce-ae88-98611ca11280 admin admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:55.520724 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00da9f34-cb16-4fce-ae88-98611ca11280 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 23:01:55.521034 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00da9f34-cb16-4fce-ae88-98611ca11280 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:55.522634 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00da9f34-cb16-4fce-ae88-98611ca11280 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:55.534827 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1301/2609] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 23:01:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 23:01:55.539168 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f41d7b0-2303-4852-8e42-267034f8d62f None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:55.548549 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f41d7b0-2303-4852-8e42-267034f8d62f None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:55.553775 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63e2660-e94b-4175-abd6-4573baa816f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:55.554423 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e2660-e94b-4175-abd6-4573baa816f6 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:55.554707 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e2660-e94b-4175-abd6-4573baa816f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:55.555007 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e2660-e94b-4175-abd6-4573baa816f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:55.555706 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63e2660-e94b-4175-abd6-4573baa816f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:55.557581 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63e2660-e94b-4175-abd6-4573baa816f6 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:55.656395 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1309/2610] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 23:01:55 2022] POST /identity/v3/projects => generated 387 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:01:55.656526 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00da9f34-cb16-4fce-ae88-98611ca11280 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:55.665791 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00da9f34-cb16-4fce-ae88-98611ca11280 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:55.723907 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffb05fd-9266-4a70-989c-b06e26ced7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:55.724576 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffb05fd-9266-4a70-989c-b06e26ced7cc admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:55.724862 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffb05fd-9266-4a70-989c-b06e26ced7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:55.725143 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffb05fd-9266-4a70-989c-b06e26ced7cc admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:55.725838 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffb05fd-9266-4a70-989c-b06e26ced7cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:55.727788 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffb05fd-9266-4a70-989c-b06e26ced7cc admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:55.831493 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1302/2611] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 23:01:55 2022] POST /identity/v3/users => generated 294 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:01:55.835557 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63e2660-e94b-4175-abd6-4573baa816f6 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:55.845943 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63e2660-e94b-4175-abd6-4573baa816f6 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:55.851094 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b8beba-74a3-4690-9251-05e79a6bb880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:55.851749 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b8beba-74a3-4690-9251-05e79a6bb880 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:55.852032 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b8beba-74a3-4690-9251-05e79a6bb880 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:55.852310 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b8beba-74a3-4690-9251-05e79a6bb880 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:55.853029 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b8beba-74a3-4690-9251-05e79a6bb880 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:55.854893 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b8beba-74a3-4690-9251-05e79a6bb880 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:55.857079 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:01:55.866427 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1310/2612] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:01:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 23:01:55.872329 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffb05fd-9266-4a70-989c-b06e26ced7cc admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:55.880419 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffb05fd-9266-4a70-989c-b06e26ced7cc admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:55.885092 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb9f732-8e9c-47b3-8757-d2c96846c5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:55.885749 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb9f732-8e9c-47b3-8757-d2c96846c5ea admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:55.886035 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb9f732-8e9c-47b3-8757-d2c96846c5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:55.886310 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb9f732-8e9c-47b3-8757-d2c96846c5ea admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:55.887101 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb9f732-8e9c-47b3-8757-d2c96846c5ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:55.888677 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb9f732-8e9c-47b3-8757-d2c96846c5ea admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:55.899052 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1303/2613] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:01:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:01:55.908035 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b8beba-74a3-4690-9251-05e79a6bb880 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:55.918404 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b8beba-74a3-4690-9251-05e79a6bb880 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:55.923385 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c166a0b-a37d-467f-86b1-e2705416db29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:55.924273 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c166a0b-a37d-467f-86b1-e2705416db29 admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:55.924970 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c166a0b-a37d-467f-86b1-e2705416db29 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:55.925238 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c166a0b-a37d-467f-86b1-e2705416db29 admin admin] PATH_INFO: `/v3/projects/7f56381501ae44e1b59b444423e1ad66/users/3eba4d5b51f7458791816b9f3d61053b/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:55.940158 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c166a0b-a37d-467f-86b1-e2705416db29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f56381501ae44e1b59b444423e1ad66, user_id=3eba4d5b51f7458791816b9f3d61053b, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:55.942850 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c166a0b-a37d-467f-86b1-e2705416db29 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:56.032160 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1311/2614] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 23:01:55 2022] PUT /identity/v3/projects/7f56381501ae44e1b59b444423e1ad66/users/3eba4d5b51f7458791816b9f3d61053b/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:01:56.044367 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a599b2e9-9f0f-4d0c-8ea0-6374dcbb9059 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:56.044654 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a599b2e9-9f0f-4d0c-8ea0-6374dcbb9059 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:56.044933 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a599b2e9-9f0f-4d0c-8ea0-6374dcbb9059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:56.051715 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:01:56.071752 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-a599b2e9-9f0f-4d0c-8ea0-6374dcbb9059 None None] MFA Rules not processed for user `3eba4d5b51f7458791816b9f3d61053b`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:01:56.074749 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-a599b2e9-9f0f-4d0c-8ea0-6374dcbb9059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:56.141330 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1304/2615] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:01:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:01:56.144897 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c166a0b-a37d-467f-86b1-e2705416db29 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:56.206080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c166a0b-a37d-467f-86b1-e2705416db29 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:56.211289 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab4b423-bf9b-44bf-ac00-3ab0ccd68ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:56.211945 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab4b423-bf9b-44bf-ac00-3ab0ccd68ed4 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:56.212230 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab4b423-bf9b-44bf-ac00-3ab0ccd68ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:56.212504 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab4b423-bf9b-44bf-ac00-3ab0ccd68ed4 admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:56.213208 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab4b423-bf9b-44bf-ac00-3ab0ccd68ed4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:56.215127 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab4b423-bf9b-44bf-ac00-3ab0ccd68ed4 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:56.347413 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1312/2616] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 23:01:56 2022] POST /identity/v3/projects => generated 385 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:01:56.352218 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a599b2e9-9f0f-4d0c-8ea0-6374dcbb9059 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:56.363584 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a599b2e9-9f0f-4d0c-8ea0-6374dcbb9059 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:56.420336 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9f30a-690b-4fcd-8bfd-1e4c2eda9487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:56.421001 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9f30a-690b-4fcd-8bfd-1e4c2eda9487 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:56.421284 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9f30a-690b-4fcd-8bfd-1e4c2eda9487 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:56.421559 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9f30a-690b-4fcd-8bfd-1e4c2eda9487 admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:56.422249 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d9f30a-690b-4fcd-8bfd-1e4c2eda9487 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:56.424196 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d9f30a-690b-4fcd-8bfd-1e4c2eda9487 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:56.517506 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1305/2617] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 23:01:56 2022] POST /identity/v3/users => generated 299 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 23:01:56.519706 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab4b423-bf9b-44bf-ac00-3ab0ccd68ed4 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:56.530010 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab4b423-bf9b-44bf-ac00-3ab0ccd68ed4 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:56.534897 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84451ced-ea2b-4772-b6d4-57c84c601b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:56.535575 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84451ced-ea2b-4772-b6d4-57c84c601b5c admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:56.535894 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84451ced-ea2b-4772-b6d4-57c84c601b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:56.536170 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84451ced-ea2b-4772-b6d4-57c84c601b5c admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:56.536905 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84451ced-ea2b-4772-b6d4-57c84c601b5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:56.539168 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84451ced-ea2b-4772-b6d4-57c84c601b5c admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:56.550317 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1313/2618] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:01:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 23:01:56.554916 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9f30a-690b-4fcd-8bfd-1e4c2eda9487 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:56.565208 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9f30a-690b-4fcd-8bfd-1e4c2eda9487 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:56.570260 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df512add-eef9-402b-b632-5905ef7c3a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:56.570970 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df512add-eef9-402b-b632-5905ef7c3a17 admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:56.571254 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df512add-eef9-402b-b632-5905ef7c3a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:56.571530 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df512add-eef9-402b-b632-5905ef7c3a17 admin admin] PATH_INFO: `/v3/projects/bea2274b75db4b1cb268d9a59512f505/users/4aa7909b42784fa397608309fa0a05f7/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:56.588057 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df512add-eef9-402b-b632-5905ef7c3a17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bea2274b75db4b1cb268d9a59512f505, user_id=4aa7909b42784fa397608309fa0a05f7, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:56.590693 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df512add-eef9-402b-b632-5905ef7c3a17 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:56.674988 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1306/2619] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 23:01:56 2022] PUT /identity/v3/projects/bea2274b75db4b1cb268d9a59512f505/users/4aa7909b42784fa397608309fa0a05f7/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:01:56.678966 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84451ced-ea2b-4772-b6d4-57c84c601b5c admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:56.688993 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84451ced-ea2b-4772-b6d4-57c84c601b5c admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:56.747272 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9a7610-c089-4313-a796-561155632b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:56.747940 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a7610-c089-4313-a796-561155632b2d admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:56.748224 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a7610-c089-4313-a796-561155632b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:56.748502 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a7610-c089-4313-a796-561155632b2d admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:56.749250 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9a7610-c089-4313-a796-561155632b2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:56.750923 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9a7610-c089-4313-a796-561155632b2d admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:56.761255 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1314/2620] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:01:56 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:01:56.765707 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df512add-eef9-402b-b632-5905ef7c3a17 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:56.776798 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df512add-eef9-402b-b632-5905ef7c3a17 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:56.781542 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c80719c-2c41-423c-9d5f-72611f02c3c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:56.783868 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c80719c-2c41-423c-9d5f-72611f02c3c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:56.783868 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c80719c-2c41-423c-9d5f-72611f02c3c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:56.783868 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c80719c-2c41-423c-9d5f-72611f02c3c4 admin admin] PATH_INFO: `/v3/projects/bea2274b75db4b1cb268d9a59512f505/users/4aa7909b42784fa397608309fa0a05f7/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:56.784987 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c80719c-2c41-423c-9d5f-72611f02c3c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bea2274b75db4b1cb268d9a59512f505, user_id=4aa7909b42784fa397608309fa0a05f7, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:56.787633 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c80719c-2c41-423c-9d5f-72611f02c3c4 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:56.797864 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1307/2621] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 23:01:56 2022] PUT /identity/v3/projects/bea2274b75db4b1cb268d9a59512f505/users/4aa7909b42784fa397608309fa0a05f7/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 23:01:56.808471 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce834b8-80ab-41de-9feb-719b0be1f56a None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:56.816698 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce834b8-80ab-41de-9feb-719b0be1f56a None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:56.816698 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce834b8-80ab-41de-9feb-719b0be1f56a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:56.835859 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-8ce834b8-80ab-41de-9feb-719b0be1f56a None None] MFA Rules not processed for user `4aa7909b42784fa397608309fa0a05f7`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:01:56.837078 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-8ce834b8-80ab-41de-9feb-719b0be1f56a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:56.869983 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:01:56.905595 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1315/2622] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:01:56 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 23:01:56.916255 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c80719c-2c41-423c-9d5f-72611f02c3c4 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:56.917934 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-1c80719c-2c41-423c-9d5f-72611f02c3c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:56.975473 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:01:56.980614 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c80719c-2c41-423c-9d5f-72611f02c3c4 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:56.984345 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04906db4-9272-473b-ba9a-83857668e2d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:56.984994 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04906db4-9272-473b-ba9a-83857668e2d5 service cinder] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:56.985280 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04906db4-9272-473b-ba9a-83857668e2d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:56.985561 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04906db4-9272-473b-ba9a-83857668e2d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:56.995025 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04906db4-9272-473b-ba9a-83857668e2d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:56.996441 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04906db4-9272-473b-ba9a-83857668e2d5 service cinder] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:57.006446 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1308/2623] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:01:56 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:01:58.264039 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce834b8-80ab-41de-9feb-719b0be1f56a None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:58.327142 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce834b8-80ab-41de-9feb-719b0be1f56a None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:58.331802 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aca21f-bfd6-4750-a3a2-b0709c3248d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:58.332677 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aca21f-bfd6-4750-a3a2-b0709c3248d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:58.332965 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aca21f-bfd6-4750-a3a2-b0709c3248d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:58.333311 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aca21f-bfd6-4750-a3a2-b0709c3248d1 admin admin] PATH_INFO: `/v3/users/3eba4d5b51f7458791816b9f3d61053b` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:58.335082 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aca21f-bfd6-4750-a3a2-b0709c3248d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3eba4d5b51f7458791816b9f3d61053b)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:58.337062 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aca21f-bfd6-4750-a3a2-b0709c3248d1 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:58.539685 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-f8aca21f-bfd6-4750-a3a2-b0709c3248d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3eba4d5b51f7458791816b9f3d61053b'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:58.546519 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-f8aca21f-bfd6-4750-a3a2-b0709c3248d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3eba4d5b51f7458791816b9f3d61053b'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:58.547206 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-f8aca21f-bfd6-4750-a3a2-b0709c3248d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3eba4d5b51f7458791816b9f3d61053b'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:58.560197 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-f8aca21f-bfd6-4750-a3a2-b0709c3248d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3eba4d5b51f7458791816b9f3d61053b'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:58.634495 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-f8aca21f-bfd6-4750-a3a2-b0709c3248d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3eba4d5b51f7458791816b9f3d61053b'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:58.640992 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1316/2624] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 23:01:58 2022] DELETE /identity/v3/users/3eba4d5b51f7458791816b9f3d61053b => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:01:58.645558 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd0459-a8e6-4978-8f0d-f98fb6a3ef4a None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:58.646016 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd0459-a8e6-4978-8f0d-f98fb6a3ef4a None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:58.646150 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd0459-a8e6-4978-8f0d-f98fb6a3ef4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:58.664197 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-a0fd0459-a8e6-4978-8f0d-f98fb6a3ef4a None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:01:58.665422 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-a0fd0459-a8e6-4978-8f0d-f98fb6a3ef4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:58.724729 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1309/2625] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:01:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:01:58.734687 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aca21f-bfd6-4750-a3a2-b0709c3248d1 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:58.736152 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-f8aca21f-bfd6-4750-a3a2-b0709c3248d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:58.790303 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:01:58.794047 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aca21f-bfd6-4750-a3a2-b0709c3248d1 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:58.797808 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea8d27e-262a-438c-a906-cd145e156072 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:58.798456 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea8d27e-262a-438c-a906-cd145e156072 service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:58.798746 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea8d27e-262a-438c-a906-cd145e156072 service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:58.799054 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea8d27e-262a-438c-a906-cd145e156072 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:58.808493 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea8d27e-262a-438c-a906-cd145e156072 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:58.809989 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea8d27e-262a-438c-a906-cd145e156072 service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:58.820358 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1317/2626] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:01:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:01:59.320158 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd0459-a8e6-4978-8f0d-f98fb6a3ef4a None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:59.322031 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-a0fd0459-a8e6-4978-8f0d-f98fb6a3ef4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:01:59.335600 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd0459-a8e6-4978-8f0d-f98fb6a3ef4a None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:59.340212 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca15ea4-529c-4013-82ae-51513d1fbb61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:59.341625 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca15ea4-529c-4013-82ae-51513d1fbb61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:59.342119 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca15ea4-529c-4013-82ae-51513d1fbb61 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:59.342415 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca15ea4-529c-4013-82ae-51513d1fbb61 admin admin] PATH_INFO: `/v3/projects/7f56381501ae44e1b59b444423e1ad66` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:59.343746 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca15ea4-529c-4013-82ae-51513d1fbb61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f56381501ae44e1b59b444423e1ad66)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:59.346133 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca15ea4-529c-4013-82ae-51513d1fbb61 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:59.484809 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-eca15ea4-529c-4013-82ae-51513d1fbb61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f56381501ae44e1b59b444423e1ad66', 'request_id': 'req-eca15ea4-529c-4013-82ae-51513d1fbb61'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:59.490337 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-eca15ea4-529c-4013-82ae-51513d1fbb61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f56381501ae44e1b59b444423e1ad66', 'request_id': 'req-eca15ea4-529c-4013-82ae-51513d1fbb61'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:59.565717 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-eca15ea4-529c-4013-82ae-51513d1fbb61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f56381501ae44e1b59b444423e1ad66', 'request_id': 'req-eca15ea4-529c-4013-82ae-51513d1fbb61'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:59.577521 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-eca15ea4-529c-4013-82ae-51513d1fbb61 admin admin] The token cache is being invalidate because project 7f56381501ae44e1b59b444423e1ad66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:01:59.577813 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-eca15ea4-529c-4013-82ae-51513d1fbb61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:59.578558 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-eca15ea4-529c-4013-82ae-51513d1fbb61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:59.580219 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1310/2627] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 23:01:59 2022] DELETE /identity/v3/projects/7f56381501ae44e1b59b444423e1ad66 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:01:59.585041 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea8d27e-262a-438c-a906-cd145e156072 service neutron] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:01:59.595290 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea8d27e-262a-438c-a906-cd145e156072 service neutron] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:01:59.652030 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8c579d-d0db-4b0b-892c-0f9995f0261a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:01:59.652899 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c579d-d0db-4b0b-892c-0f9995f0261a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:01:59.653183 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c579d-d0db-4b0b-892c-0f9995f0261a admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:01:59.653498 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c579d-d0db-4b0b-892c-0f9995f0261a admin admin] PATH_INFO: `/v3/users/4aa7909b42784fa397608309fa0a05f7` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:01:59.654708 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8c579d-d0db-4b0b-892c-0f9995f0261a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4aa7909b42784fa397608309fa0a05f7)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:01:59.656747 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8c579d-d0db-4b0b-892c-0f9995f0261a admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:01:59.891881 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-af8c579d-d0db-4b0b-892c-0f9995f0261a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4aa7909b42784fa397608309fa0a05f7'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:59.899482 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-af8c579d-d0db-4b0b-892c-0f9995f0261a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4aa7909b42784fa397608309fa0a05f7'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:59.900171 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-af8c579d-d0db-4b0b-892c-0f9995f0261a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4aa7909b42784fa397608309fa0a05f7'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:59.913800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-af8c579d-d0db-4b0b-892c-0f9995f0261a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4aa7909b42784fa397608309fa0a05f7'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:59.992686 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-af8c579d-d0db-4b0b-892c-0f9995f0261a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4aa7909b42784fa397608309fa0a05f7'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:01:59.999108 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1318/2628] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 23:01:59 2022] DELETE /identity/v3/users/4aa7909b42784fa397608309fa0a05f7 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:00.526404 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca15ea4-529c-4013-82ae-51513d1fbb61 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:00.527368 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8c579d-d0db-4b0b-892c-0f9995f0261a admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:00.527916 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-eca15ea4-529c-4013-82ae-51513d1fbb61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:00.529057 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-af8c579d-d0db-4b0b-892c-0f9995f0261a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:00.608426 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8c579d-d0db-4b0b-892c-0f9995f0261a admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:00.612414 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca15ea4-529c-4013-82ae-51513d1fbb61 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:00.613434 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5699eb-0449-4c90-9a05-54f7d96130f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:00.614388 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5699eb-0449-4c90-9a05-54f7d96130f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:00.614742 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5699eb-0449-4c90-9a05-54f7d96130f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:00.616135 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5699eb-0449-4c90-9a05-54f7d96130f7 admin admin] PATH_INFO: `/v3/projects/bea2274b75db4b1cb268d9a59512f505` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:00.617320 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5699eb-0449-4c90-9a05-54f7d96130f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bea2274b75db4b1cb268d9a59512f505)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:00.619440 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5699eb-0449-4c90-9a05-54f7d96130f7 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:00.632188 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:00.639287 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:00.663481 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69e871-2481-4e1f-8b3e-18cd5f71f47c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:00.664145 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69e871-2481-4e1f-8b3e-18cd5f71f47c service neutron] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:00.664432 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69e871-2481-4e1f-8b3e-18cd5f71f47c service neutron] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:00.664821 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69e871-2481-4e1f-8b3e-18cd5f71f47c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:00.666505 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-ce69e871-2481-4e1f-8b3e-18cd5f71f47c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:00.679295 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce69e871-2481-4e1f-8b3e-18cd5f71f47c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:00.680750 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce69e871-2481-4e1f-8b3e-18cd5f71f47c service neutron] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:00.690693 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1311/2629] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:02:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:02:00.742496 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-7b5699eb-0449-4c90-9a05-54f7d96130f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bea2274b75db4b1cb268d9a59512f505', 'request_id': 'req-7b5699eb-0449-4c90-9a05-54f7d96130f7'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:00.750229 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-7b5699eb-0449-4c90-9a05-54f7d96130f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bea2274b75db4b1cb268d9a59512f505', 'request_id': 'req-7b5699eb-0449-4c90-9a05-54f7d96130f7'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:00.829081 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-7b5699eb-0449-4c90-9a05-54f7d96130f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bea2274b75db4b1cb268d9a59512f505', 'request_id': 'req-7b5699eb-0449-4c90-9a05-54f7d96130f7'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:00.843801 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-7b5699eb-0449-4c90-9a05-54f7d96130f7 admin admin] The token cache is being invalidate because project bea2274b75db4b1cb268d9a59512f505 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:02:00.844219 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-7b5699eb-0449-4c90-9a05-54f7d96130f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:00.845078 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-7b5699eb-0449-4c90-9a05-54f7d96130f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:00.851162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1319/2630] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 23:02:00 2022] DELETE /identity/v3/projects/bea2274b75db4b1cb268d9a59512f505 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:00.854418 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa6686d-2faf-45d8-af5a-b1321c8a9a19 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:00.854418 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa6686d-2faf-45d8-af5a-b1321c8a9a19 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:00.854418 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa6686d-2faf-45d8-af5a-b1321c8a9a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:00.873394 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-0aa6686d-2faf-45d8-af5a-b1321c8a9a19 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:02:00.874629 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-0aa6686d-2faf-45d8-af5a-b1321c8a9a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:00.943099 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1312/2631] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:02:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 23:02:00.943831 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e470cf2f-7137-4a32-96ee-1343fd91b2b0 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:00.944253 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e470cf2f-7137-4a32-96ee-1343fd91b2b0 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:00.944655 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e470cf2f-7137-4a32-96ee-1343fd91b2b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:00.964769 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-e470cf2f-7137-4a32-96ee-1343fd91b2b0 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:02:00.965983 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-e470cf2f-7137-4a32-96ee-1343fd91b2b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:00.974091 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1320/2632] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:02:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:02:00.979487 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa6686d-2faf-45d8-af5a-b1321c8a9a19 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:00.989192 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa6686d-2faf-45d8-af5a-b1321c8a9a19 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:00.994617 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588aca2b-027d-4531-9c02-030b505d30eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:00.995394 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588aca2b-027d-4531-9c02-030b505d30eb admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:00.995665 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588aca2b-027d-4531-9c02-030b505d30eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:00.995945 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588aca2b-027d-4531-9c02-030b505d30eb admin admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:00.996676 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588aca2b-027d-4531-9c02-030b505d30eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 23:02:00.996997 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588aca2b-027d-4531-9c02-030b505d30eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:00.998592 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588aca2b-027d-4531-9c02-030b505d30eb admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:01.011001 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1313/2633] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 23:02:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 23:02:01.017701 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e470cf2f-7137-4a32-96ee-1343fd91b2b0 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:01.025732 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e470cf2f-7137-4a32-96ee-1343fd91b2b0 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:01.030444 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a89770-1303-4c81-b8fe-31a65d057768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:01.031127 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a89770-1303-4c81-b8fe-31a65d057768 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:01.031410 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a89770-1303-4c81-b8fe-31a65d057768 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:01.031685 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a89770-1303-4c81-b8fe-31a65d057768 admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:01.032374 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a89770-1303-4c81-b8fe-31a65d057768 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:01.034332 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a89770-1303-4c81-b8fe-31a65d057768 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:01.122040 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1321/2634] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 23:02:01 2022] POST /identity/v3/projects => generated 381 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 23:02:01.126147 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588aca2b-027d-4531-9c02-030b505d30eb admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:01.137087 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588aca2b-027d-4531-9c02-030b505d30eb admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:01.194983 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ed8bbd-cb7b-4f24-9c08-188a1112fc83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:01.195578 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed8bbd-cb7b-4f24-9c08-188a1112fc83 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:01.195863 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed8bbd-cb7b-4f24-9c08-188a1112fc83 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:01.196139 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed8bbd-cb7b-4f24-9c08-188a1112fc83 admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:01.197092 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ed8bbd-cb7b-4f24-9c08-188a1112fc83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:01.199044 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ed8bbd-cb7b-4f24-9c08-188a1112fc83 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:01.327840 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1314/2635] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 23:02:01 2022] POST /identity/v3/users => generated 291 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:02:01.332002 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a89770-1303-4c81-b8fe-31a65d057768 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:01.343538 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a89770-1303-4c81-b8fe-31a65d057768 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:01.348507 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ddfe32-a349-46cf-a699-a9ec6378a304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:01.349259 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ddfe32-a349-46cf-a699-a9ec6378a304 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:01.349641 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ddfe32-a349-46cf-a699-a9ec6378a304 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:01.349975 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ddfe32-a349-46cf-a699-a9ec6378a304 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:01.350826 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ddfe32-a349-46cf-a699-a9ec6378a304 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:01.356711 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ddfe32-a349-46cf-a699-a9ec6378a304 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:01.368796 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1322/2636] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:02:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:02:01.373065 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ed8bbd-cb7b-4f24-9c08-188a1112fc83 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:01.384172 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ed8bbd-cb7b-4f24-9c08-188a1112fc83 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:01.389183 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a8778-04e6-4987-bde2-3a606810a9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:01.389852 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a8778-04e6-4987-bde2-3a606810a9c6 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:01.390138 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a8778-04e6-4987-bde2-3a606810a9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:01.390420 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a8778-04e6-4987-bde2-3a606810a9c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:01.391188 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605a8778-04e6-4987-bde2-3a606810a9c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:01.392789 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605a8778-04e6-4987-bde2-3a606810a9c6 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:01.406761 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1315/2637] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:02:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:02:01.407898 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ddfe32-a349-46cf-a699-a9ec6378a304 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:01.418011 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ddfe32-a349-46cf-a699-a9ec6378a304 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:01.422799 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00213050-ddfe-4536-9a36-9401aab0b64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:01.423717 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00213050-ddfe-4536-9a36-9401aab0b64b admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:01.423997 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00213050-ddfe-4536-9a36-9401aab0b64b admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:01.424276 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00213050-ddfe-4536-9a36-9401aab0b64b admin admin] PATH_INFO: `/v3/projects/6ca667b8b228452ea5a23673b3f027bb/users/b6c5a00fb8464b6bb615b35afe02fae2/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:01.439283 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00213050-ddfe-4536-9a36-9401aab0b64b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ca667b8b228452ea5a23673b3f027bb, user_id=b6c5a00fb8464b6bb615b35afe02fae2, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:01.441964 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00213050-ddfe-4536-9a36-9401aab0b64b admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:01.539021 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1323/2638] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 23:02:01 2022] PUT /identity/v3/projects/6ca667b8b228452ea5a23673b3f027bb/users/b6c5a00fb8464b6bb615b35afe02fae2/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:01.546639 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab443b3-18b4-414e-b243-6ad6bc7a98df None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:01.547219 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab443b3-18b4-414e-b243-6ad6bc7a98df None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:01.547349 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab443b3-18b4-414e-b243-6ad6bc7a98df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:01.583188 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-bab443b3-18b4-414e-b243-6ad6bc7a98df None None] MFA Rules not processed for user `b6c5a00fb8464b6bb615b35afe02fae2`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:02:01.584605 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-bab443b3-18b4-414e-b243-6ad6bc7a98df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:01.614055 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:01.653274 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1316/2639] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:02:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:02:01.657805 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00213050-ddfe-4536-9a36-9401aab0b64b admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:01.727775 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00213050-ddfe-4536-9a36-9401aab0b64b admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:01.733042 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e69b275-3e56-4b58-83ea-bf4e633e97ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:01.733746 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69b275-3e56-4b58-83ea-bf4e633e97ee admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:01.734026 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69b275-3e56-4b58-83ea-bf4e633e97ee admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:01.734329 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69b275-3e56-4b58-83ea-bf4e633e97ee admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:01.735227 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e69b275-3e56-4b58-83ea-bf4e633e97ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:01.737272 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e69b275-3e56-4b58-83ea-bf4e633e97ee admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:01.920300 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1324/2640] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 23:02:01 2022] POST /identity/v3/projects => generated 381 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:02:01.921899 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab443b3-18b4-414e-b243-6ad6bc7a98df None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:01.934601 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab443b3-18b4-414e-b243-6ad6bc7a98df None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:02.007755 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1251b4e-7106-4fa2-9f05-172de5731639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:02.008286 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1251b4e-7106-4fa2-9f05-172de5731639 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:02.008573 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1251b4e-7106-4fa2-9f05-172de5731639 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:02.008853 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1251b4e-7106-4fa2-9f05-172de5731639 admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:02.009555 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1251b4e-7106-4fa2-9f05-172de5731639 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:02.011507 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1251b4e-7106-4fa2-9f05-172de5731639 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:02.092558 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1317/2641] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 23:02:01 2022] POST /identity/v3/users => generated 297 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:02:02.093867 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e69b275-3e56-4b58-83ea-bf4e633e97ee admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:02.105419 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e69b275-3e56-4b58-83ea-bf4e633e97ee admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:02.110254 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a608cb3-b0a5-4178-844f-93e11365183d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:02.110932 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a608cb3-b0a5-4178-844f-93e11365183d admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:02.111224 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a608cb3-b0a5-4178-844f-93e11365183d admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:02.111507 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a608cb3-b0a5-4178-844f-93e11365183d admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:02.112271 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a608cb3-b0a5-4178-844f-93e11365183d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:02.113891 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a608cb3-b0a5-4178-844f-93e11365183d admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:02.125516 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1325/2642] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:02:02.131963 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1251b4e-7106-4fa2-9f05-172de5731639 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:02.142117 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1251b4e-7106-4fa2-9f05-172de5731639 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:02.147092 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b27d3b-3adc-4fbc-a402-04c8842a22d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:02.148088 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b27d3b-3adc-4fbc-a402-04c8842a22d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:02.148382 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b27d3b-3adc-4fbc-a402-04c8842a22d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:02.148661 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b27d3b-3adc-4fbc-a402-04c8842a22d8 admin admin] PATH_INFO: `/v3/projects/1054a950f1324168baf600948dbfe4e1/users/34bcd714eee347a7aa6edcc44a1b9ef4/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:02.164843 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b27d3b-3adc-4fbc-a402-04c8842a22d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1054a950f1324168baf600948dbfe4e1, user_id=34bcd714eee347a7aa6edcc44a1b9ef4, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:02.167657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b27d3b-3adc-4fbc-a402-04c8842a22d8 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:02.248780 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1318/2643] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 23:02:02 2022] PUT /identity/v3/projects/1054a950f1324168baf600948dbfe4e1/users/34bcd714eee347a7aa6edcc44a1b9ef4/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:02.252994 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a608cb3-b0a5-4178-844f-93e11365183d admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:02.265483 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a608cb3-b0a5-4178-844f-93e11365183d admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:02.274333 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:02.342604 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f024144a-c052-447d-ab5d-6de4b147fe5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:02.343319 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f024144a-c052-447d-ab5d-6de4b147fe5a admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:02.343608 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f024144a-c052-447d-ab5d-6de4b147fe5a admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:02.343890 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f024144a-c052-447d-ab5d-6de4b147fe5a admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:02.344630 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f024144a-c052-447d-ab5d-6de4b147fe5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:02.346230 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f024144a-c052-447d-ab5d-6de4b147fe5a admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:02.358434 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1326/2644] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:02:02.362631 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b27d3b-3adc-4fbc-a402-04c8842a22d8 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:02.373952 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b27d3b-3adc-4fbc-a402-04c8842a22d8 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:02.379007 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b4bb08-37b8-42a6-bb63-be11c603fe25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:02.379897 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b4bb08-37b8-42a6-bb63-be11c603fe25 admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:02.380187 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b4bb08-37b8-42a6-bb63-be11c603fe25 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:02.380470 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b4bb08-37b8-42a6-bb63-be11c603fe25 admin admin] PATH_INFO: `/v3/projects/1054a950f1324168baf600948dbfe4e1/users/34bcd714eee347a7aa6edcc44a1b9ef4/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:02.382508 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b4bb08-37b8-42a6-bb63-be11c603fe25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1054a950f1324168baf600948dbfe4e1, user_id=34bcd714eee347a7aa6edcc44a1b9ef4, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:02.385208 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b4bb08-37b8-42a6-bb63-be11c603fe25 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:02.398161 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1319/2645] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 23:02:02 2022] PUT /identity/v3/projects/1054a950f1324168baf600948dbfe4e1/users/34bcd714eee347a7aa6edcc44a1b9ef4/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:02.408616 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6bb27c-8180-40fe-af53-0f4764c08240 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:02.408904 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6bb27c-8180-40fe-af53-0f4764c08240 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:02.410281 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6bb27c-8180-40fe-af53-0f4764c08240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:02.437301 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-8f6bb27c-8180-40fe-af53-0f4764c08240 None None] MFA Rules not processed for user `34bcd714eee347a7aa6edcc44a1b9ef4`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:02:02.438572 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-8f6bb27c-8180-40fe-af53-0f4764c08240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:02.529131 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1327/2646] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:02:02 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:02:02.541324 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b4bb08-37b8-42a6-bb63-be11c603fe25 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:02.545638 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-68b4bb08-37b8-42a6-bb63-be11c603fe25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:02.613686 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b4bb08-37b8-42a6-bb63-be11c603fe25 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:02.618242 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7da5d1-9eb7-49b1-aa9a-db9bcafcdee4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:02.619104 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7da5d1-9eb7-49b1-aa9a-db9bcafcdee4 service cinder] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:02.619537 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7da5d1-9eb7-49b1-aa9a-db9bcafcdee4 service cinder] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:02.619964 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7da5d1-9eb7-49b1-aa9a-db9bcafcdee4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:02.630537 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7da5d1-9eb7-49b1-aa9a-db9bcafcdee4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:02.632727 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7da5d1-9eb7-49b1-aa9a-db9bcafcdee4 service cinder] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:02.643718 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1320/2647] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:02:02 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:02:03.181624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6bb27c-8180-40fe-af53-0f4764c08240 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:03.250004 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6bb27c-8180-40fe-af53-0f4764c08240 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:03.255164 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba221c-a8b8-45b0-a030-90f14cbc4150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:03.256831 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba221c-a8b8-45b0-a030-90f14cbc4150 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:03.257117 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba221c-a8b8-45b0-a030-90f14cbc4150 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:03.257393 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba221c-a8b8-45b0-a030-90f14cbc4150 admin admin] PATH_INFO: `/v3/users/b6c5a00fb8464b6bb615b35afe02fae2` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:03.258626 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bba221c-a8b8-45b0-a030-90f14cbc4150 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6c5a00fb8464b6bb615b35afe02fae2)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:03.260630 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bba221c-a8b8-45b0-a030-90f14cbc4150 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:03.450833 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:03.460269 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-7bba221c-a8b8-45b0-a030-90f14cbc4150 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6c5a00fb8464b6bb615b35afe02fae2'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:03.467220 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-7bba221c-a8b8-45b0-a030-90f14cbc4150 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6c5a00fb8464b6bb615b35afe02fae2'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:03.467892 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-7bba221c-a8b8-45b0-a030-90f14cbc4150 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6c5a00fb8464b6bb615b35afe02fae2'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:03.481028 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-7bba221c-a8b8-45b0-a030-90f14cbc4150 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6c5a00fb8464b6bb615b35afe02fae2'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:03.558029 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-7bba221c-a8b8-45b0-a030-90f14cbc4150 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6c5a00fb8464b6bb615b35afe02fae2'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:03.564774 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1328/2648] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 23:02:03 2022] DELETE /identity/v3/users/b6c5a00fb8464b6bb615b35afe02fae2 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:03.569193 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed6585-014d-4855-b587-fa502a40706d None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:03.569486 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed6585-014d-4855-b587-fa502a40706d None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:03.569790 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed6585-014d-4855-b587-fa502a40706d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:03.594483 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-2bed6585-014d-4855-b587-fa502a40706d None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:02:03.596475 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-2bed6585-014d-4855-b587-fa502a40706d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:03.619729 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:03.673930 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1321/2649] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:02:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 23:02:03.689778 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba221c-a8b8-45b0-a030-90f14cbc4150 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:03.694945 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-7bba221c-a8b8-45b0-a030-90f14cbc4150 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:03.835228 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba221c-a8b8-45b0-a030-90f14cbc4150 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:03.839288 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e801be-17c3-46df-813b-1747ff476a25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:03.840003 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e801be-17c3-46df-813b-1747ff476a25 service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:03.840482 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e801be-17c3-46df-813b-1747ff476a25 service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:03.840618 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e801be-17c3-46df-813b-1747ff476a25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:03.855474 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e801be-17c3-46df-813b-1747ff476a25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:03.857673 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e801be-17c3-46df-813b-1747ff476a25 service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:03.871810 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1329/2650] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:02:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:02:04.531358 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bed6585-014d-4855-b587-fa502a40706d None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:04.533040 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-2bed6585-014d-4855-b587-fa502a40706d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:04.549048 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bed6585-014d-4855-b587-fa502a40706d None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:04.554520 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b97300-974d-42d1-b9bc-722f09e42725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:04.555590 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b97300-974d-42d1-b9bc-722f09e42725 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:04.555945 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b97300-974d-42d1-b9bc-722f09e42725 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:04.556163 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b97300-974d-42d1-b9bc-722f09e42725 admin admin] PATH_INFO: `/v3/projects/6ca667b8b228452ea5a23673b3f027bb` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:04.557440 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b97300-974d-42d1-b9bc-722f09e42725 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ca667b8b228452ea5a23673b3f027bb)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:04.559551 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b97300-974d-42d1-b9bc-722f09e42725 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:04.682196 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-c7b97300-974d-42d1-b9bc-722f09e42725 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ca667b8b228452ea5a23673b3f027bb', 'request_id': 'req-c7b97300-974d-42d1-b9bc-722f09e42725'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:04.688241 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-c7b97300-974d-42d1-b9bc-722f09e42725 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ca667b8b228452ea5a23673b3f027bb', 'request_id': 'req-c7b97300-974d-42d1-b9bc-722f09e42725'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:04.752101 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-c7b97300-974d-42d1-b9bc-722f09e42725 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ca667b8b228452ea5a23673b3f027bb', 'request_id': 'req-c7b97300-974d-42d1-b9bc-722f09e42725'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:04.764396 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-c7b97300-974d-42d1-b9bc-722f09e42725 admin admin] The token cache is being invalidate because project 6ca667b8b228452ea5a23673b3f027bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:02:04.764808 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-c7b97300-974d-42d1-b9bc-722f09e42725 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:04.765442 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-c7b97300-974d-42d1-b9bc-722f09e42725 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:04.767040 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1322/2651] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 23:02:04 2022] DELETE /identity/v3/projects/6ca667b8b228452ea5a23673b3f027bb => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:04.771162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e801be-17c3-46df-813b-1747ff476a25 service neutron] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:04.781817 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e801be-17c3-46df-813b-1747ff476a25 service neutron] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:04.791577 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b97300-974d-42d1-b9bc-722f09e42725 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:04.793223 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-c7b97300-974d-42d1-b9bc-722f09e42725 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:04.842860 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7d0ff2-aed2-4c6f-a80d-840a408f38d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:04.843722 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d0ff2-aed2-4c6f-a80d-840a408f38d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:04.844007 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d0ff2-aed2-4c6f-a80d-840a408f38d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:04.844286 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d0ff2-aed2-4c6f-a80d-840a408f38d2 admin admin] PATH_INFO: `/v3/users/34bcd714eee347a7aa6edcc44a1b9ef4` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:04.845483 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7d0ff2-aed2-4c6f-a80d-840a408f38d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34bcd714eee347a7aa6edcc44a1b9ef4)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:04.847433 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7d0ff2-aed2-4c6f-a80d-840a408f38d2 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:04.863951 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b97300-974d-42d1-b9bc-722f09e42725 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:04.869810 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4061b-0d88-4b99-97ba-1e8d1dbf7d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:04.871163 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4061b-0d88-4b99-97ba-1e8d1dbf7d14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:04.871457 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4061b-0d88-4b99-97ba-1e8d1dbf7d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:04.871743 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4061b-0d88-4b99-97ba-1e8d1dbf7d14 admin admin] PATH_INFO: `/v3/users/8c11dc2f4c764dc58458d9226efc878e` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:04.872998 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d4061b-0d88-4b99-97ba-1e8d1dbf7d14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c11dc2f4c764dc58458d9226efc878e)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:04.875078 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d4061b-0d88-4b99-97ba-1e8d1dbf7d14 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:05.081711 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-2b7d0ff2-aed2-4c6f-a80d-840a408f38d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34bcd714eee347a7aa6edcc44a1b9ef4'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:05.089338 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-2b7d0ff2-aed2-4c6f-a80d-840a408f38d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34bcd714eee347a7aa6edcc44a1b9ef4'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:05.090160 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-2b7d0ff2-aed2-4c6f-a80d-840a408f38d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34bcd714eee347a7aa6edcc44a1b9ef4'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:05.090663 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-49d4061b-0d88-4b99-97ba-1e8d1dbf7d14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c11dc2f4c764dc58458d9226efc878e'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:05.091616 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:05.092492 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:05.101438 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-49d4061b-0d88-4b99-97ba-1e8d1dbf7d14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c11dc2f4c764dc58458d9226efc878e'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:05.103166 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-49d4061b-0d88-4b99-97ba-1e8d1dbf7d14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c11dc2f4c764dc58458d9226efc878e'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:05.105960 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-2b7d0ff2-aed2-4c6f-a80d-840a408f38d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34bcd714eee347a7aa6edcc44a1b9ef4'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:05.120459 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-49d4061b-0d88-4b99-97ba-1e8d1dbf7d14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c11dc2f4c764dc58458d9226efc878e'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:05.121166 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-49d4061b-0d88-4b99-97ba-1e8d1dbf7d14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c11dc2f4c764dc58458d9226efc878e'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:05.186480 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-2b7d0ff2-aed2-4c6f-a80d-840a408f38d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34bcd714eee347a7aa6edcc44a1b9ef4'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:05.193076 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1330/2652] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 23:02:04 2022] DELETE /identity/v3/users/34bcd714eee347a7aa6edcc44a1b9ef4 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:05.266419 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1323/2653] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 23:02:04 2022] DELETE /identity/v3/users/8c11dc2f4c764dc58458d9226efc878e => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 23:02:05.550072 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7d0ff2-aed2-4c6f-a80d-840a408f38d2 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:05.551940 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-2b7d0ff2-aed2-4c6f-a80d-840a408f38d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:05.623528 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7d0ff2-aed2-4c6f-a80d-840a408f38d2 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:05.640850 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4061b-0d88-4b99-97ba-1e8d1dbf7d14 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:05.642436 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-49d4061b-0d88-4b99-97ba-1e8d1dbf7d14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:05.683739 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec701710-4696-4ea3-be36-2fe9034119b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:05.684604 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec701710-4696-4ea3-be36-2fe9034119b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:05.684891 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec701710-4696-4ea3-be36-2fe9034119b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:05.685164 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec701710-4696-4ea3-be36-2fe9034119b0 admin admin] PATH_INFO: `/v3/projects/224678a5f1db4416b8175d315a4e4a0d` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:05.686339 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec701710-4696-4ea3-be36-2fe9034119b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=224678a5f1db4416b8175d315a4e4a0d)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:05.688247 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec701710-4696-4ea3-be36-2fe9034119b0 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:05.713572 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4061b-0d88-4b99-97ba-1e8d1dbf7d14 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:05.718209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210fb084-6f1a-4090-9ecd-fab56e1099a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:05.719150 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210fb084-6f1a-4090-9ecd-fab56e1099a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:05.719436 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210fb084-6f1a-4090-9ecd-fab56e1099a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:05.719734 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210fb084-6f1a-4090-9ecd-fab56e1099a5 admin admin] PATH_INFO: `/v3/projects/1054a950f1324168baf600948dbfe4e1` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:05.720920 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210fb084-6f1a-4090-9ecd-fab56e1099a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1054a950f1324168baf600948dbfe4e1)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:05.722845 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210fb084-6f1a-4090-9ecd-fab56e1099a5 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:05.795814 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-ec701710-4696-4ea3-be36-2fe9034119b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '224678a5f1db4416b8175d315a4e4a0d', 'request_id': 'req-ec701710-4696-4ea3-be36-2fe9034119b0'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:05.801320 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-ec701710-4696-4ea3-be36-2fe9034119b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '224678a5f1db4416b8175d315a4e4a0d', 'request_id': 'req-ec701710-4696-4ea3-be36-2fe9034119b0'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:05.835027 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-210fb084-6f1a-4090-9ecd-fab56e1099a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1054a950f1324168baf600948dbfe4e1', 'request_id': 'req-210fb084-6f1a-4090-9ecd-fab56e1099a5'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:05.840435 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-210fb084-6f1a-4090-9ecd-fab56e1099a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1054a950f1324168baf600948dbfe4e1', 'request_id': 'req-210fb084-6f1a-4090-9ecd-fab56e1099a5'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:05.908501 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-ec701710-4696-4ea3-be36-2fe9034119b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '224678a5f1db4416b8175d315a4e4a0d', 'request_id': 'req-ec701710-4696-4ea3-be36-2fe9034119b0'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:05.919900 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-ec701710-4696-4ea3-be36-2fe9034119b0 admin admin] The token cache is being invalidate because project 224678a5f1db4416b8175d315a4e4a0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:02:05.920189 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-ec701710-4696-4ea3-be36-2fe9034119b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:05.920798 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-ec701710-4696-4ea3-be36-2fe9034119b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:05.922409 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1331/2654] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 23:02:05 2022] DELETE /identity/v3/projects/224678a5f1db4416b8175d315a4e4a0d => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:05.927226 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294860b0-d4e4-472e-be62-bc331f608bee None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:05.927507 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294860b0-d4e4-472e-be62-bc331f608bee None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:05.927781 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294860b0-d4e4-472e-be62-bc331f608bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:05.945442 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-294860b0-d4e4-472e-be62-bc331f608bee None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:02:05.946634 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-294860b0-d4e4-472e-be62-bc331f608bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:05.951822 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:05.965558 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-210fb084-6f1a-4090-9ecd-fab56e1099a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1054a950f1324168baf600948dbfe4e1', 'request_id': 'req-210fb084-6f1a-4090-9ecd-fab56e1099a5'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:05.977288 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-210fb084-6f1a-4090-9ecd-fab56e1099a5 admin admin] The token cache is being invalidate because project 1054a950f1324168baf600948dbfe4e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:02:05.977633 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-210fb084-6f1a-4090-9ecd-fab56e1099a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:05.978353 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-210fb084-6f1a-4090-9ecd-fab56e1099a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:05.980208 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1324/2655] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 23:02:05 2022] DELETE /identity/v3/projects/1054a950f1324168baf600948dbfe4e1 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 23:02:05.984926 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a182043f-75f7-4508-b23c-fd523035d29d None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:05.985215 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a182043f-75f7-4508-b23c-fd523035d29d None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:05.985496 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a182043f-75f7-4508-b23c-fd523035d29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:06.004011 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-a182043f-75f7-4508-b23c-fd523035d29d None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:02:06.005222 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-a182043f-75f7-4508-b23c-fd523035d29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:06.008168 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1332/2656] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:02:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 23:02:06.015706 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1325/2657] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:02:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 23:02:06.015854 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c03ea8-4688-4ca1-a648-953cee0698b7 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:06.016164 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c03ea8-4688-4ca1-a648-953cee0698b7 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:06.016537 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c03ea8-4688-4ca1-a648-953cee0698b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:06.023473 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7228767a-aea1-47ac-8abe-90f2c7591796 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:06.023751 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7228767a-aea1-47ac-8abe-90f2c7591796 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:06.024035 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7228767a-aea1-47ac-8abe-90f2c7591796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:06.034322 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-84c03ea8-4688-4ca1-a648-953cee0698b7 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:02:06.035552 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-84c03ea8-4688-4ca1-a648-953cee0698b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:06.042699 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-7228767a-aea1-47ac-8abe-90f2c7591796 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:02:06.043522 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1333/2658] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:02:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 23:02:06.043948 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-7228767a-aea1-47ac-8abe-90f2c7591796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:06.047528 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c03ea8-4688-4ca1-a648-953cee0698b7 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:06.054119 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1326/2659] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:02:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 23:02:06.056886 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c03ea8-4688-4ca1-a648-953cee0698b7 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:06.058237 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7228767a-aea1-47ac-8abe-90f2c7591796 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:06.061962 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f089876f-9c9c-463c-89f1-db0be9a81b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:06.062132 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f089876f-9c9c-463c-89f1-db0be9a81b4e admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:06.062404 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f089876f-9c9c-463c-89f1-db0be9a81b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:06.062678 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f089876f-9c9c-463c-89f1-db0be9a81b4e admin admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:06.063433 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f089876f-9c9c-463c-89f1-db0be9a81b4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 23:02:06.064202 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f089876f-9c9c-463c-89f1-db0be9a81b4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:06.065314 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f089876f-9c9c-463c-89f1-db0be9a81b4e admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:06.067910 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7228767a-aea1-47ac-8abe-90f2c7591796 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:06.072872 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508264b1-e2aa-4181-9e8d-3927d3ab855e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:06.073561 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508264b1-e2aa-4181-9e8d-3927d3ab855e admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:06.073848 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508264b1-e2aa-4181-9e8d-3927d3ab855e admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:06.074126 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508264b1-e2aa-4181-9e8d-3927d3ab855e admin admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:06.074877 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508264b1-e2aa-4181-9e8d-3927d3ab855e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 23:02:06.075188 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508264b1-e2aa-4181-9e8d-3927d3ab855e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:06.076771 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508264b1-e2aa-4181-9e8d-3927d3ab855e admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:06.077978 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1334/2660] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 23:02:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 23:02:06.082162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f089876f-9c9c-463c-89f1-db0be9a81b4e admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:06.088983 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1327/2661] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 23:02:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 23:02:06.091893 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f089876f-9c9c-463c-89f1-db0be9a81b4e admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:06.093682 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508264b1-e2aa-4181-9e8d-3927d3ab855e admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:06.096704 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006929f3-864c-49fd-a192-8fab7462dd3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:06.097349 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006929f3-864c-49fd-a192-8fab7462dd3f admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:06.097634 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006929f3-864c-49fd-a192-8fab7462dd3f admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:06.097912 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006929f3-864c-49fd-a192-8fab7462dd3f admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:06.098626 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006929f3-864c-49fd-a192-8fab7462dd3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:06.100586 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006929f3-864c-49fd-a192-8fab7462dd3f admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:06.103824 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508264b1-e2aa-4181-9e8d-3927d3ab855e admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:06.108799 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c0dd51-8d8c-46b3-a71c-1c81e7f6c694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:06.109435 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0dd51-8d8c-46b3-a71c-1c81e7f6c694 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:06.109718 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0dd51-8d8c-46b3-a71c-1c81e7f6c694 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:06.109987 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0dd51-8d8c-46b3-a71c-1c81e7f6c694 admin admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:06.110707 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c0dd51-8d8c-46b3-a71c-1c81e7f6c694 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:06.112655 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c0dd51-8d8c-46b3-a71c-1c81e7f6c694 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:06.200266 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1335/2662] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 23:02:06 2022] POST /identity/v3/projects => generated 373 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 23:02:06.204156 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006929f3-864c-49fd-a192-8fab7462dd3f admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:06.227033 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1328/2663] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 23:02:06 2022] POST /identity/v3/projects => generated 383 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:02:06.228041 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c0dd51-8d8c-46b3-a71c-1c81e7f6c694 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:06.237476 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:06.273939 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006929f3-864c-49fd-a192-8fab7462dd3f admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:06.294004 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c0dd51-8d8c-46b3-a71c-1c81e7f6c694 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:06.299045 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9164f97c-1b5f-4dcf-960a-05c75bf0b2a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:06.299723 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164f97c-1b5f-4dcf-960a-05c75bf0b2a3 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:06.300014 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164f97c-1b5f-4dcf-960a-05c75bf0b2a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:06.300290 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9164f97c-1b5f-4dcf-960a-05c75bf0b2a3 admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:06.300986 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9164f97c-1b5f-4dcf-960a-05c75bf0b2a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:06.303469 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9164f97c-1b5f-4dcf-960a-05c75bf0b2a3 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:06.338527 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df853927-82f6-46f2-8d0b-d95afa87c80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:06.339402 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df853927-82f6-46f2-8d0b-d95afa87c80e admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:06.339689 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df853927-82f6-46f2-8d0b-d95afa87c80e admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:06.340022 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df853927-82f6-46f2-8d0b-d95afa87c80e admin admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:06.340715 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df853927-82f6-46f2-8d0b-d95afa87c80e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:06.342723 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df853927-82f6-46f2-8d0b-d95afa87c80e admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:06.375022 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1329/2664] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 23:02:06 2022] POST /identity/v3/users => generated 292 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 23:02:06.379066 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9164f97c-1b5f-4dcf-960a-05c75bf0b2a3 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:06.388860 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9164f97c-1b5f-4dcf-960a-05c75bf0b2a3 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:06.394477 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e284c4-f372-4af9-99c1-b8b114b78b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:06.395233 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e284c4-f372-4af9-99c1-b8b114b78b22 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:06.397121 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e284c4-f372-4af9-99c1-b8b114b78b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:06.397121 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e284c4-f372-4af9-99c1-b8b114b78b22 admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:06.397121 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e284c4-f372-4af9-99c1-b8b114b78b22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:06.398370 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e284c4-f372-4af9-99c1-b8b114b78b22 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:06.408694 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1330/2665] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 23:02:06.413268 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e284c4-f372-4af9-99c1-b8b114b78b22 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:06.426364 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e284c4-f372-4af9-99c1-b8b114b78b22 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:06.431720 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d9671a-8a53-482f-890c-12ec3360771d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:06.432379 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9671a-8a53-482f-890c-12ec3360771d admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:06.432667 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9671a-8a53-482f-890c-12ec3360771d admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:06.432949 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9671a-8a53-482f-890c-12ec3360771d admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:06.433671 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d9671a-8a53-482f-890c-12ec3360771d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:06.435284 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d9671a-8a53-482f-890c-12ec3360771d admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:06.445617 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1331/2666] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 23:02:06.449329 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1336/2667] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 23:02:06 2022] POST /identity/v3/users => generated 287 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 23:02:06.449637 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d9671a-8a53-482f-890c-12ec3360771d admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:06.453293 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df853927-82f6-46f2-8d0b-d95afa87c80e admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:06.459912 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d9671a-8a53-482f-890c-12ec3360771d admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:06.463577 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df853927-82f6-46f2-8d0b-d95afa87c80e admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:06.464592 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6197c3a8-6d4d-4753-8e73-d8dfa8789d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:06.465461 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6197c3a8-6d4d-4753-8e73-d8dfa8789d7a admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:06.465746 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6197c3a8-6d4d-4753-8e73-d8dfa8789d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:06.466021 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6197c3a8-6d4d-4753-8e73-d8dfa8789d7a admin admin] PATH_INFO: `/v3/projects/ab5f235b30c54dedb41679fb94a7dfa2/users/ec073dc9f5ff4010a01b93901b9ad33d/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:06.468500 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25596edc-9ed5-446b-a4ce-b707668a43e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:06.469239 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25596edc-9ed5-446b-a4ce-b707668a43e6 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:06.469592 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25596edc-9ed5-446b-a4ce-b707668a43e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:06.469940 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25596edc-9ed5-446b-a4ce-b707668a43e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:06.470749 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25596edc-9ed5-446b-a4ce-b707668a43e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:06.472372 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25596edc-9ed5-446b-a4ce-b707668a43e6 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:06.482003 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6197c3a8-6d4d-4753-8e73-d8dfa8789d7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab5f235b30c54dedb41679fb94a7dfa2, user_id=ec073dc9f5ff4010a01b93901b9ad33d, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:06.484623 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1337/2668] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 23:02:06.484790 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6197c3a8-6d4d-4753-8e73-d8dfa8789d7a admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:06.488518 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25596edc-9ed5-446b-a4ce-b707668a43e6 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:06.499045 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25596edc-9ed5-446b-a4ce-b707668a43e6 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:06.503953 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e05d75-9ce7-4fbc-8f9a-f24c995ed3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:06.504617 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e05d75-9ce7-4fbc-8f9a-f24c995ed3c6 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:06.504902 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e05d75-9ce7-4fbc-8f9a-f24c995ed3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:06.505178 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e05d75-9ce7-4fbc-8f9a-f24c995ed3c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:06.505936 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e05d75-9ce7-4fbc-8f9a-f24c995ed3c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:06.507571 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e05d75-9ce7-4fbc-8f9a-f24c995ed3c6 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:06.518019 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1338/2669] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 23:02:06.521999 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e05d75-9ce7-4fbc-8f9a-f24c995ed3c6 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:06.531192 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e05d75-9ce7-4fbc-8f9a-f24c995ed3c6 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:06.535814 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dcb0d6-3e41-4529-8128-859b63e0febb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:06.536647 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dcb0d6-3e41-4529-8128-859b63e0febb admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:06.537671 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dcb0d6-3e41-4529-8128-859b63e0febb admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:06.537947 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dcb0d6-3e41-4529-8128-859b63e0febb admin admin] PATH_INFO: `/v3/projects/9963be03a1e84a698a67707390b7240a/users/08d24284ad884d0699814870576e1c7f/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:06.551809 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dcb0d6-3e41-4529-8128-859b63e0febb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9963be03a1e84a698a67707390b7240a, user_id=08d24284ad884d0699814870576e1c7f, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:06.555077 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dcb0d6-3e41-4529-8128-859b63e0febb admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:06.569136 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1332/2670] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 23:02:06 2022] PUT /identity/v3/projects/ab5f235b30c54dedb41679fb94a7dfa2/users/ec073dc9f5ff4010a01b93901b9ad33d/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 23:02:06.632359 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1339/2671] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 23:02:06 2022] PUT /identity/v3/projects/9963be03a1e84a698a67707390b7240a/users/08d24284ad884d0699814870576e1c7f/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 23:02:06.636469 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae283755-aeb2-4fbb-9bc2-44ef47886340 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:06.636789 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae283755-aeb2-4fbb-9bc2-44ef47886340 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:06.637077 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae283755-aeb2-4fbb-9bc2-44ef47886340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:06.642336 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e9ca22-924a-490b-aef7-76f1216c5d1f None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:06.642626 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e9ca22-924a-490b-aef7-76f1216c5d1f None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:06.642934 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e9ca22-924a-490b-aef7-76f1216c5d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:06.656531 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:06.664422 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-ae283755-aeb2-4fbb-9bc2-44ef47886340 None None] MFA Rules not processed for user `ec073dc9f5ff4010a01b93901b9ad33d`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:02:06.665655 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-ae283755-aeb2-4fbb-9bc2-44ef47886340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:06.671508 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-92e9ca22-924a-490b-aef7-76f1216c5d1f None None] MFA Rules not processed for user `08d24284ad884d0699814870576e1c7f`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:02:06.672699 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-92e9ca22-924a-490b-aef7-76f1216c5d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:06.732508 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1333/2672] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:02:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 23:02:06.737343 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1340/2673] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:02:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 23:02:06.743440 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae283755-aeb2-4fbb-9bc2-44ef47886340 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:06.744838 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-ae283755-aeb2-4fbb-9bc2-44ef47886340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:06.748772 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e9ca22-924a-490b-aef7-76f1216c5d1f None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:06.750117 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-92e9ca22-924a-490b-aef7-76f1216c5d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:06.801697 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae283755-aeb2-4fbb-9bc2-44ef47886340 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:06.805514 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f7aa7-6b92-4ca7-8adc-06a25f217ed2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:06.806161 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f7aa7-6b92-4ca7-8adc-06a25f217ed2 service cinder] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:06.806447 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f7aa7-6b92-4ca7-8adc-06a25f217ed2 service cinder] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:06.806728 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f7aa7-6b92-4ca7-8adc-06a25f217ed2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:06.815173 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605f7aa7-6b92-4ca7-8adc-06a25f217ed2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:06.816625 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605f7aa7-6b92-4ca7-8adc-06a25f217ed2 service cinder] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:06.822959 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e9ca22-924a-490b-aef7-76f1216c5d1f None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:06.825648 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1334/2674] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:02:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:02:06.828936 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916e6c3c-348f-4c9a-9a39-a10288a7f618 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bfdb655a2a44b33ab724daaf73a294b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:06.829653 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e6c3c-348f-4c9a-9a39-a10288a7f618 service nova] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:06.830025 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e6c3c-348f-4c9a-9a39-a10288a7f618 service nova] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:06.830419 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e6c3c-348f-4c9a-9a39-a10288a7f618 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:06.839410 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916e6c3c-348f-4c9a-9a39-a10288a7f618 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:06.841172 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916e6c3c-348f-4c9a-9a39-a10288a7f618 service nova] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:06.843742 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:06.850859 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1341/2675] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:02:06 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 23:02:16.901235 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=83,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:16.957515 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=8 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:18.014676 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f7aa7-6b92-4ca7-8adc-06a25f217ed2 service cinder] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:18.107230 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f7aa7-6b92-4ca7-8adc-06a25f217ed2 service cinder] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:18.118038 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a2c43d-cf8a-4801-ae08-e95addd1def5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:18.119363 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2c43d-cf8a-4801-ae08-e95addd1def5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:18.119704 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2c43d-cf8a-4801-ae08-e95addd1def5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:18.120034 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2c43d-cf8a-4801-ae08-e95addd1def5 admin admin] PATH_INFO: `/v3/users/08d24284ad884d0699814870576e1c7f` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:18.121594 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a2c43d-cf8a-4801-ae08-e95addd1def5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08d24284ad884d0699814870576e1c7f)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:18.123869 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a2c43d-cf8a-4801-ae08-e95addd1def5 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:18.412834 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-d7a2c43d-cf8a-4801-ae08-e95addd1def5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08d24284ad884d0699814870576e1c7f'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:18.419112 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-d7a2c43d-cf8a-4801-ae08-e95addd1def5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08d24284ad884d0699814870576e1c7f'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:18.420135 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-d7a2c43d-cf8a-4801-ae08-e95addd1def5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08d24284ad884d0699814870576e1c7f'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:18.433032 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-d7a2c43d-cf8a-4801-ae08-e95addd1def5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08d24284ad884d0699814870576e1c7f'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:18.433677 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-d7a2c43d-cf8a-4801-ae08-e95addd1def5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08d24284ad884d0699814870576e1c7f'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:18.543448 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1335/2676] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 23:02:18 2022] DELETE /identity/v3/users/08d24284ad884d0699814870576e1c7f => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:18.547984 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c7a4e-b68b-4d2e-b8df-1b99b6be0d29 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:18.548446 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c7a4e-b68b-4d2e-b8df-1b99b6be0d29 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:18.548577 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c7a4e-b68b-4d2e-b8df-1b99b6be0d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:18.566212 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-d81c7a4e-b68b-4d2e-b8df-1b99b6be0d29 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:02:18.567470 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-d81c7a4e-b68b-4d2e-b8df-1b99b6be0d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:18.628228 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1342/2677] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:02:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:02:18.638151 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a2c43d-cf8a-4801-ae08-e95addd1def5 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:18.639687 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-d7a2c43d-cf8a-4801-ae08-e95addd1def5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:18.703228 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a2c43d-cf8a-4801-ae08-e95addd1def5 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:18.706927 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b22086b-a681-4827-8a3e-4b82f0fa4576 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:18.707599 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b22086b-a681-4827-8a3e-4b82f0fa4576 service neutron] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:18.707892 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b22086b-a681-4827-8a3e-4b82f0fa4576 service neutron] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:18.708173 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b22086b-a681-4827-8a3e-4b82f0fa4576 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:18.718352 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b22086b-a681-4827-8a3e-4b82f0fa4576 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:18.719785 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b22086b-a681-4827-8a3e-4b82f0fa4576 service neutron] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:18.730479 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1336/2678] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:02:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 23:02:19.179683 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81c7a4e-b68b-4d2e-b8df-1b99b6be0d29 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:19.185797 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-d81c7a4e-b68b-4d2e-b8df-1b99b6be0d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:19.222198 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81c7a4e-b68b-4d2e-b8df-1b99b6be0d29 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:19.227528 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4442c96-3438-4a5b-bc29-46c62d4cbe79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:19.228440 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4442c96-3438-4a5b-bc29-46c62d4cbe79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:19.228794 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4442c96-3438-4a5b-bc29-46c62d4cbe79 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:19.229534 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4442c96-3438-4a5b-bc29-46c62d4cbe79 admin admin] PATH_INFO: `/v3/projects/9963be03a1e84a698a67707390b7240a` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:19.230712 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4442c96-3438-4a5b-bc29-46c62d4cbe79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9963be03a1e84a698a67707390b7240a)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:19.232752 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4442c96-3438-4a5b-bc29-46c62d4cbe79 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:19.413693 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-e4442c96-3438-4a5b-bc29-46c62d4cbe79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9963be03a1e84a698a67707390b7240a', 'request_id': 'req-e4442c96-3438-4a5b-bc29-46c62d4cbe79'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:19.420416 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-e4442c96-3438-4a5b-bc29-46c62d4cbe79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9963be03a1e84a698a67707390b7240a', 'request_id': 'req-e4442c96-3438-4a5b-bc29-46c62d4cbe79'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:19.501070 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-e4442c96-3438-4a5b-bc29-46c62d4cbe79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9963be03a1e84a698a67707390b7240a', 'request_id': 'req-e4442c96-3438-4a5b-bc29-46c62d4cbe79'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:19.524337 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-e4442c96-3438-4a5b-bc29-46c62d4cbe79 admin admin] The token cache is being invalidate because project 9963be03a1e84a698a67707390b7240a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:02:19.524700 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-e4442c96-3438-4a5b-bc29-46c62d4cbe79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:19.525389 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-e4442c96-3438-4a5b-bc29-46c62d4cbe79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:19.528159 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1343/2679] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 23:02:19 2022] DELETE /identity/v3/projects/9963be03a1e84a698a67707390b7240a => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:19.533641 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb7926b-659f-4ce5-a267-076068826e81 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:19.534123 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb7926b-659f-4ce5-a267-076068826e81 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:19.534238 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb7926b-659f-4ce5-a267-076068826e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:19.559030 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-9cb7926b-659f-4ce5-a267-076068826e81 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:02:19.560315 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-9cb7926b-659f-4ce5-a267-076068826e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:19.570866 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4442c96-3438-4a5b-bc29-46c62d4cbe79 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:19.572520 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-e4442c96-3438-4a5b-bc29-46c62d4cbe79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:19.578527 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:19.624385 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1337/2680] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:02:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:02:19.633765 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c8ee1e-94a9-4d40-b41c-dcbc365ce8f5 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:19.634056 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c8ee1e-94a9-4d40-b41c-dcbc365ce8f5 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:19.634337 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c8ee1e-94a9-4d40-b41c-dcbc365ce8f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:19.637078 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4442c96-3438-4a5b-bc29-46c62d4cbe79 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:19.640846 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b09a17-acd4-45b2-836e-24f44b358b16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:19.641538 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b09a17-acd4-45b2-836e-24f44b358b16 service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:19.641824 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b09a17-acd4-45b2-836e-24f44b358b16 service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:19.642104 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b09a17-acd4-45b2-836e-24f44b358b16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:19.644011 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-87b09a17-acd4-45b2-836e-24f44b358b16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:19.653698 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-73c8ee1e-94a9-4d40-b41c-dcbc365ce8f5 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:02:19.654934 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-73c8ee1e-94a9-4d40-b41c-dcbc365ce8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:19.662882 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1338/2681] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:02:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:02:19.667006 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c8ee1e-94a9-4d40-b41c-dcbc365ce8f5 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:19.675983 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c8ee1e-94a9-4d40-b41c-dcbc365ce8f5 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:19.681317 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d148cf5b-4428-4ba9-9173-ab2edcfff741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:19.681964 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d148cf5b-4428-4ba9-9173-ab2edcfff741 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:19.682253 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d148cf5b-4428-4ba9-9173-ab2edcfff741 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:19.682529 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d148cf5b-4428-4ba9-9173-ab2edcfff741 admin admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:19.683333 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d148cf5b-4428-4ba9-9173-ab2edcfff741 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 23:02:19.683647 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d148cf5b-4428-4ba9-9173-ab2edcfff741 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:19.685308 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d148cf5b-4428-4ba9-9173-ab2edcfff741 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:19.687243 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:19.698003 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1339/2682] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 23:02:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 23:02:19.702244 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d148cf5b-4428-4ba9-9173-ab2edcfff741 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:19.711922 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d148cf5b-4428-4ba9-9173-ab2edcfff741 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:19.713056 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b09a17-acd4-45b2-836e-24f44b358b16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:19.714439 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b09a17-acd4-45b2-836e-24f44b358b16 service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:19.716846 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e110012c-8511-4920-8c6c-a45779d2a2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:19.717540 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e110012c-8511-4920-8c6c-a45779d2a2c6 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:19.717822 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e110012c-8511-4920-8c6c-a45779d2a2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:19.718097 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e110012c-8511-4920-8c6c-a45779d2a2c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:19.718935 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e110012c-8511-4920-8c6c-a45779d2a2c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:19.720930 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e110012c-8511-4920-8c6c-a45779d2a2c6 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:19.725348 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1344/2683] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:02:19 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 23:02:19.851383 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1340/2684] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 23:02:19 2022] POST /identity/v3/projects => generated 373 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 23:02:19.855761 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b09a17-acd4-45b2-836e-24f44b358b16 service neutron] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:19.867096 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b09a17-acd4-45b2-836e-24f44b358b16 service neutron] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:19.923646 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d70f41-6ee0-4f7c-8cd0-d04fda42936f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:19.924467 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d70f41-6ee0-4f7c-8cd0-d04fda42936f admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:19.924752 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d70f41-6ee0-4f7c-8cd0-d04fda42936f admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:19.925065 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d70f41-6ee0-4f7c-8cd0-d04fda42936f admin admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:19.925752 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d70f41-6ee0-4f7c-8cd0-d04fda42936f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:19.928304 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d70f41-6ee0-4f7c-8cd0-d04fda42936f admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:20.045360 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1345/2685] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 23:02:19 2022] POST /identity/v3/users => generated 287 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:02:20.046598 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e110012c-8511-4920-8c6c-a45779d2a2c6 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:20.063259 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e110012c-8511-4920-8c6c-a45779d2a2c6 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:20.070743 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77aadd2-f918-45e0-a142-1d90644b8309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:20.072809 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77aadd2-f918-45e0-a142-1d90644b8309 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:20.073958 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77aadd2-f918-45e0-a142-1d90644b8309 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:20.074888 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77aadd2-f918-45e0-a142-1d90644b8309 admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:20.075873 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77aadd2-f918-45e0-a142-1d90644b8309 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:20.078424 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77aadd2-f918-45e0-a142-1d90644b8309 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:20.094404 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1341/2686] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:02:20.095726 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d70f41-6ee0-4f7c-8cd0-d04fda42936f admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:20.106756 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d70f41-6ee0-4f7c-8cd0-d04fda42936f admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:20.111382 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3072ee-165a-4acf-aedc-3ab39a50d69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:20.112042 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3072ee-165a-4acf-aedc-3ab39a50d69b admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:20.112324 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3072ee-165a-4acf-aedc-3ab39a50d69b admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:20.112596 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3072ee-165a-4acf-aedc-3ab39a50d69b admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:20.113323 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3072ee-165a-4acf-aedc-3ab39a50d69b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:20.115602 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3072ee-165a-4acf-aedc-3ab39a50d69b admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:20.129458 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1346/2687] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:02:20.130361 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77aadd2-f918-45e0-a142-1d90644b8309 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:20.140472 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77aadd2-f918-45e0-a142-1d90644b8309 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:20.145318 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aac7a4-5020-4da3-8bc3-5e5b816028e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:20.146208 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aac7a4-5020-4da3-8bc3-5e5b816028e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:20.146502 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aac7a4-5020-4da3-8bc3-5e5b816028e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:20.146793 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aac7a4-5020-4da3-8bc3-5e5b816028e0 admin admin] PATH_INFO: `/v3/projects/7cf2fec2943c4371844d205ae919d1ac/users/4bc0d32aeb384ac08160a20a07a885fe/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:20.164176 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aac7a4-5020-4da3-8bc3-5e5b816028e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cf2fec2943c4371844d205ae919d1ac, user_id=4bc0d32aeb384ac08160a20a07a885fe, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:20.167655 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aac7a4-5020-4da3-8bc3-5e5b816028e0 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:20.279919 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1342/2688] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 23:02:20 2022] PUT /identity/v3/projects/7cf2fec2943c4371844d205ae919d1ac/users/4bc0d32aeb384ac08160a20a07a885fe/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 23:02:20.292527 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c3474-931b-42b6-a756-cc07d1be7118 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:20.293328 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c3474-931b-42b6-a756-cc07d1be7118 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:20.293859 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c3474-931b-42b6-a756-cc07d1be7118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:20.325260 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-6d2c3474-931b-42b6-a756-cc07d1be7118 None None] MFA Rules not processed for user `4bc0d32aeb384ac08160a20a07a885fe`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:02:20.326502 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-6d2c3474-931b-42b6-a756-cc07d1be7118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:20.400212 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1347/2689] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:02:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:02:20.411287 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aac7a4-5020-4da3-8bc3-5e5b816028e0 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:20.413691 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-33aac7a4-5020-4da3-8bc3-5e5b816028e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:20.511743 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aac7a4-5020-4da3-8bc3-5e5b816028e0 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:20.521430 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d25a1-849c-41ed-bfe7-15e2b47b044e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bfdb655a2a44b33ab724daaf73a294b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:20.522189 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d25a1-849c-41ed-bfe7-15e2b47b044e service nova] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:20.522685 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d25a1-849c-41ed-bfe7-15e2b47b044e service nova] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:20.522835 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d25a1-849c-41ed-bfe7-15e2b47b044e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:20.533432 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432d25a1-849c-41ed-bfe7-15e2b47b044e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:20.535066 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432d25a1-849c-41ed-bfe7-15e2b47b044e service nova] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:20.546944 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1343/2690] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:02:20 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:02:20.588911 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2c3474-931b-42b6-a756-cc07d1be7118 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:20.595307 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-6d2c3474-931b-42b6-a756-cc07d1be7118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:20.614390 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d25a1-849c-41ed-bfe7-15e2b47b044e service nova] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:20.618249 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-432d25a1-849c-41ed-bfe7-15e2b47b044e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:20.668251 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:20.686262 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:20.715573 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2c3474-931b-42b6-a756-cc07d1be7118 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:20.716320 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50107ace-a8c0-4109-a7a9-95ff73dab53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:20.717430 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50107ace-a8c0-4109-a7a9-95ff73dab53a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:20.717875 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50107ace-a8c0-4109-a7a9-95ff73dab53a admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:20.718318 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50107ace-a8c0-4109-a7a9-95ff73dab53a admin admin] PATH_INFO: `/v3/users/e3b076c33f784569a1fafd49f7c2f314` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:20.720136 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50107ace-a8c0-4109-a7a9-95ff73dab53a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3b076c33f784569a1fafd49f7c2f314)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:20.722235 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50107ace-a8c0-4109-a7a9-95ff73dab53a admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:20.731532 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d25a1-849c-41ed-bfe7-15e2b47b044e service nova] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:20.737143 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d386e-be3d-4117-b471-3dbdfca6952f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:20.738177 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d386e-be3d-4117-b471-3dbdfca6952f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:20.738986 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d386e-be3d-4117-b471-3dbdfca6952f admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:20.739413 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d386e-be3d-4117-b471-3dbdfca6952f admin admin] PATH_INFO: `/v3/users/3439a666c26740f9b27d78cd721f5fa6` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:20.741216 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7d386e-be3d-4117-b471-3dbdfca6952f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3439a666c26740f9b27d78cd721f5fa6)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:20.745099 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7d386e-be3d-4117-b471-3dbdfca6952f admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:21.016087 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-50107ace-a8c0-4109-a7a9-95ff73dab53a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3b076c33f784569a1fafd49f7c2f314'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:21.017454 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-7f7d386e-be3d-4117-b471-3dbdfca6952f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3439a666c26740f9b27d78cd721f5fa6'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:21.023564 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-50107ace-a8c0-4109-a7a9-95ff73dab53a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3b076c33f784569a1fafd49f7c2f314'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:21.024270 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-50107ace-a8c0-4109-a7a9-95ff73dab53a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3b076c33f784569a1fafd49f7c2f314'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:21.024762 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-7f7d386e-be3d-4117-b471-3dbdfca6952f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3439a666c26740f9b27d78cd721f5fa6'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:21.025519 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-7f7d386e-be3d-4117-b471-3dbdfca6952f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3439a666c26740f9b27d78cd721f5fa6'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:21.036738 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-50107ace-a8c0-4109-a7a9-95ff73dab53a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3b076c33f784569a1fafd49f7c2f314'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:21.038071 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-7f7d386e-be3d-4117-b471-3dbdfca6952f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3439a666c26740f9b27d78cd721f5fa6'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:21.038924 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-7f7d386e-be3d-4117-b471-3dbdfca6952f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3439a666c26740f9b27d78cd721f5fa6'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:21.123396 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-50107ace-a8c0-4109-a7a9-95ff73dab53a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3b076c33f784569a1fafd49f7c2f314'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:21.130107 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1348/2691] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 23:02:20 2022] DELETE /identity/v3/users/e3b076c33f784569a1fafd49f7c2f314 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:21.133800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50107ace-a8c0-4109-a7a9-95ff73dab53a admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:21.133800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-50107ace-a8c0-4109-a7a9-95ff73dab53a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:21.204795 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50107ace-a8c0-4109-a7a9-95ff73dab53a admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:21.210192 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0e39f0-fae0-4d20-8022-2e1c705b5457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:21.210563 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e39f0-fae0-4d20-8022-2e1c705b5457 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:21.210878 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e39f0-fae0-4d20-8022-2e1c705b5457 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:21.211148 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e39f0-fae0-4d20-8022-2e1c705b5457 admin admin] PATH_INFO: `/v3/users/4bc0d32aeb384ac08160a20a07a885fe` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:21.212350 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0e39f0-fae0-4d20-8022-2e1c705b5457 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bc0d32aeb384ac08160a20a07a885fe)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:21.214430 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0e39f0-fae0-4d20-8022-2e1c705b5457 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:21.245448 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1344/2692] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 23:02:20 2022] DELETE /identity/v3/users/3439a666c26740f9b27d78cd721f5fa6 => generated 0 bytes in 632 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:21.246875 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16369568-26fb-4395-8933-83ad3268070c None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:21.247173 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16369568-26fb-4395-8933-83ad3268070c None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:21.247509 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16369568-26fb-4395-8933-83ad3268070c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:21.265340 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-16369568-26fb-4395-8933-83ad3268070c None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:02:21.266620 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-16369568-26fb-4395-8933-83ad3268070c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:21.335368 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1345/2693] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:02:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:02:21.350208 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68276b8a-a181-418b-b1eb-5e4140a91e59 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:21.350208 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68276b8a-a181-418b-b1eb-5e4140a91e59 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:21.350208 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68276b8a-a181-418b-b1eb-5e4140a91e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:21.357328 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-68276b8a-a181-418b-b1eb-5e4140a91e59 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:02:21.358603 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-68276b8a-a181-418b-b1eb-5e4140a91e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:21.367774 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1346/2694] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:02:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:02:21.368703 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68276b8a-a181-418b-b1eb-5e4140a91e59 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:21.370127 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-68276b8a-a181-418b-b1eb-5e4140a91e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:21.384382 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68276b8a-a181-418b-b1eb-5e4140a91e59 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:21.389985 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3987d94-2fee-4439-ad72-442c0414566c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:21.390978 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3987d94-2fee-4439-ad72-442c0414566c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:21.391328 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3987d94-2fee-4439-ad72-442c0414566c admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:21.391673 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3987d94-2fee-4439-ad72-442c0414566c admin admin] PATH_INFO: `/v3/users/ec073dc9f5ff4010a01b93901b9ad33d` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:21.393344 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3987d94-2fee-4439-ad72-442c0414566c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec073dc9f5ff4010a01b93901b9ad33d)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:21.395901 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3987d94-2fee-4439-ad72-442c0414566c admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:21.610343 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-4e0e39f0-fae0-4d20-8022-2e1c705b5457 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bc0d32aeb384ac08160a20a07a885fe'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:21.617923 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-4e0e39f0-fae0-4d20-8022-2e1c705b5457 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bc0d32aeb384ac08160a20a07a885fe'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:21.618701 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-4e0e39f0-fae0-4d20-8022-2e1c705b5457 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bc0d32aeb384ac08160a20a07a885fe'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:21.625181 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:21.632435 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-4e0e39f0-fae0-4d20-8022-2e1c705b5457 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bc0d32aeb384ac08160a20a07a885fe'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:21.688490 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:21.695294 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-a3987d94-2fee-4439-ad72-442c0414566c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec073dc9f5ff4010a01b93901b9ad33d'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:21.702106 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-a3987d94-2fee-4439-ad72-442c0414566c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec073dc9f5ff4010a01b93901b9ad33d'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:21.702957 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-a3987d94-2fee-4439-ad72-442c0414566c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec073dc9f5ff4010a01b93901b9ad33d'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:21.715785 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-a3987d94-2fee-4439-ad72-442c0414566c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec073dc9f5ff4010a01b93901b9ad33d'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:21.716469 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-a3987d94-2fee-4439-ad72-442c0414566c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec073dc9f5ff4010a01b93901b9ad33d'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:21.725518 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-4e0e39f0-fae0-4d20-8022-2e1c705b5457 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bc0d32aeb384ac08160a20a07a885fe'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:21.733061 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1349/2695] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 23:02:21 2022] DELETE /identity/v3/users/4bc0d32aeb384ac08160a20a07a885fe => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:21.734073 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0e39f0-fae0-4d20-8022-2e1c705b5457 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:21.735533 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-4e0e39f0-fae0-4d20-8022-2e1c705b5457 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:21.793080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0e39f0-fae0-4d20-8022-2e1c705b5457 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:21.796695 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93a3dd6-3bf4-41df-a8c5-e54eefb549cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:21.797345 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a3dd6-3bf4-41df-a8c5-e54eefb549cf service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:21.797663 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a3dd6-3bf4-41df-a8c5-e54eefb549cf service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:21.797944 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a3dd6-3bf4-41df-a8c5-e54eefb549cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:21.799807 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-c93a3dd6-3bf4-41df-a8c5-e54eefb549cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:21.806986 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1347/2696] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 23:02:21 2022] DELETE /identity/v3/users/ec073dc9f5ff4010a01b93901b9ad33d => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:21.807766 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3987d94-2fee-4439-ad72-442c0414566c admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:21.809201 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-a3987d94-2fee-4439-ad72-442c0414566c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:21.872818 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3987d94-2fee-4439-ad72-442c0414566c admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:21.875863 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93a3dd6-3bf4-41df-a8c5-e54eefb549cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:21.876623 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c883ecd3-e532-4015-ad3a-ee77958342ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:21.877315 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93a3dd6-3bf4-41df-a8c5-e54eefb549cf service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:21.877441 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883ecd3-e532-4015-ad3a-ee77958342ee service neutron] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:21.877622 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883ecd3-e532-4015-ad3a-ee77958342ee service neutron] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:21.877906 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883ecd3-e532-4015-ad3a-ee77958342ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:21.879815 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-c883ecd3-e532-4015-ad3a-ee77958342ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:21.888176 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1350/2697] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:02:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:02:21.889562 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca1caa-0bb8-4f20-95b0-a85bdcecacb9 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:21.889869 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca1caa-0bb8-4f20-95b0-a85bdcecacb9 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:21.890131 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca1caa-0bb8-4f20-95b0-a85bdcecacb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:21.910734 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-7aca1caa-0bb8-4f20-95b0-a85bdcecacb9 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:02:21.912010 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-7aca1caa-0bb8-4f20-95b0-a85bdcecacb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:21.976955 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c883ecd3-e532-4015-ad3a-ee77958342ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:21.978483 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c883ecd3-e532-4015-ad3a-ee77958342ee service neutron] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:21.990125 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1348/2698] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:02:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:02:21.994850 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180b3cec-9db7-46f5-82f2-3f6e612ec10e None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:21.994850 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180b3cec-9db7-46f5-82f2-3f6e612ec10e None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:21.994850 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180b3cec-9db7-46f5-82f2-3f6e612ec10e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:21.999478 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1351/2699] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:02:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:02:22.011652 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aca1caa-0bb8-4f20-95b0-a85bdcecacb9 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:22.017434 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-180b3cec-9db7-46f5-82f2-3f6e612ec10e None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:02:22.018794 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-180b3cec-9db7-46f5-82f2-3f6e612ec10e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:22.023669 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-7aca1caa-0bb8-4f20-95b0-a85bdcecacb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:22.029335 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1349/2700] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:02:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:02:22.041648 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180b3cec-9db7-46f5-82f2-3f6e612ec10e None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:22.042220 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aca1caa-0bb8-4f20-95b0-a85bdcecacb9 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:22.050473 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29156065-0b03-45d5-a870-a18a7c2b0f2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:22.051157 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29156065-0b03-45d5-a870-a18a7c2b0f2f service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:22.051453 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29156065-0b03-45d5-a870-a18a7c2b0f2f service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:22.051735 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29156065-0b03-45d5-a870-a18a7c2b0f2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:22.056138 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180b3cec-9db7-46f5-82f2-3f6e612ec10e None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:22.060720 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a62b124-11da-4e8c-9697-cc647dbbe2f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:22.062288 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a62b124-11da-4e8c-9697-cc647dbbe2f6 service neutron] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:22.062578 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a62b124-11da-4e8c-9697-cc647dbbe2f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:22.062888 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a62b124-11da-4e8c-9697-cc647dbbe2f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:22.073274 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29156065-0b03-45d5-a870-a18a7c2b0f2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:22.074339 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a62b124-11da-4e8c-9697-cc647dbbe2f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:22.075009 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29156065-0b03-45d5-a870-a18a7c2b0f2f service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:22.075934 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a62b124-11da-4e8c-9697-cc647dbbe2f6 service neutron] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:22.087104 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1350/2701] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:02:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 23:02:22.091774 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1352/2702] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:02:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:02:22.507028 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a62b124-11da-4e8c-9697-cc647dbbe2f6 service neutron] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:22.508496 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-6a62b124-11da-4e8c-9697-cc647dbbe2f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:22.522686 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a62b124-11da-4e8c-9697-cc647dbbe2f6 service neutron] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:22.532212 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1b9941-590f-47c8-9188-924dbf4893fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:22.533156 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b9941-590f-47c8-9188-924dbf4893fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:22.533448 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b9941-590f-47c8-9188-924dbf4893fd admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:22.533728 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b9941-590f-47c8-9188-924dbf4893fd admin admin] PATH_INFO: `/v3/projects/a0e5f01e7bae402d9ee2697ff9ae5461` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:22.537591 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1b9941-590f-47c8-9188-924dbf4893fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0e5f01e7bae402d9ee2697ff9ae5461)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:22.538272 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1b9941-590f-47c8-9188-924dbf4893fd admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:22.644537 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29156065-0b03-45d5-a870-a18a7c2b0f2f service neutron] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:22.646342 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-29156065-0b03-45d5-a870-a18a7c2b0f2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:22.672340 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:22.680350 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29156065-0b03-45d5-a870-a18a7c2b0f2f service neutron] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:22.685858 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-da1b9941-590f-47c8-9188-924dbf4893fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0e5f01e7bae402d9ee2697ff9ae5461', 'request_id': 'req-da1b9941-590f-47c8-9188-924dbf4893fd'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:22.704447 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-da1b9941-590f-47c8-9188-924dbf4893fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0e5f01e7bae402d9ee2697ff9ae5461', 'request_id': 'req-da1b9941-590f-47c8-9188-924dbf4893fd'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:22.716526 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:22.762324 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407a0212-241c-4b2a-9820-cbcb06ffc9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:22.763223 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407a0212-241c-4b2a-9820-cbcb06ffc9c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:22.763511 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407a0212-241c-4b2a-9820-cbcb06ffc9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:22.763789 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407a0212-241c-4b2a-9820-cbcb06ffc9c8 admin admin] PATH_INFO: `/v3/projects/fe53d6ca4fa44a67ae4a039635d2c5b6` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:22.765014 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407a0212-241c-4b2a-9820-cbcb06ffc9c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe53d6ca4fa44a67ae4a039635d2c5b6)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:22.766964 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407a0212-241c-4b2a-9820-cbcb06ffc9c8 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:22.780340 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-da1b9941-590f-47c8-9188-924dbf4893fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0e5f01e7bae402d9ee2697ff9ae5461', 'request_id': 'req-da1b9941-590f-47c8-9188-924dbf4893fd'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:22.821208 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-da1b9941-590f-47c8-9188-924dbf4893fd admin admin] The token cache is being invalidate because project a0e5f01e7bae402d9ee2697ff9ae5461 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:02:22.821636 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-da1b9941-590f-47c8-9188-924dbf4893fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:22.822251 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-da1b9941-590f-47c8-9188-924dbf4893fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:22.823845 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1351/2703] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 23:02:22 2022] DELETE /identity/v3/projects/a0e5f01e7bae402d9ee2697ff9ae5461 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:22.824791 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1b9941-590f-47c8-9188-924dbf4893fd admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:22.829676 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-da1b9941-590f-47c8-9188-924dbf4893fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:22.847301 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1b9941-590f-47c8-9188-924dbf4893fd admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:22.854443 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7371ecf9-45d7-4a3c-bbb9-4b3f93e00b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:22.855776 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7371ecf9-45d7-4a3c-bbb9-4b3f93e00b37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:22.856107 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7371ecf9-45d7-4a3c-bbb9-4b3f93e00b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:22.856525 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7371ecf9-45d7-4a3c-bbb9-4b3f93e00b37 admin admin] PATH_INFO: `/v3/projects/ab5f235b30c54dedb41679fb94a7dfa2` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:22.857686 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7371ecf9-45d7-4a3c-bbb9-4b3f93e00b37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab5f235b30c54dedb41679fb94a7dfa2)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:22.860096 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7371ecf9-45d7-4a3c-bbb9-4b3f93e00b37 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:22.949259 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-407a0212-241c-4b2a-9820-cbcb06ffc9c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe53d6ca4fa44a67ae4a039635d2c5b6', 'request_id': 'req-407a0212-241c-4b2a-9820-cbcb06ffc9c8'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:22.959210 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-407a0212-241c-4b2a-9820-cbcb06ffc9c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe53d6ca4fa44a67ae4a039635d2c5b6', 'request_id': 'req-407a0212-241c-4b2a-9820-cbcb06ffc9c8'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:23.021480 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-7371ecf9-45d7-4a3c-bbb9-4b3f93e00b37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab5f235b30c54dedb41679fb94a7dfa2', 'request_id': 'req-7371ecf9-45d7-4a3c-bbb9-4b3f93e00b37'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:23.023519 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-407a0212-241c-4b2a-9820-cbcb06ffc9c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe53d6ca4fa44a67ae4a039635d2c5b6', 'request_id': 'req-407a0212-241c-4b2a-9820-cbcb06ffc9c8'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:23.039755 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-407a0212-241c-4b2a-9820-cbcb06ffc9c8 admin admin] The token cache is being invalidate because project fe53d6ca4fa44a67ae4a039635d2c5b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:02:23.040074 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-407a0212-241c-4b2a-9820-cbcb06ffc9c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:23.040857 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-407a0212-241c-4b2a-9820-cbcb06ffc9c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:23.041654 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-7371ecf9-45d7-4a3c-bbb9-4b3f93e00b37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab5f235b30c54dedb41679fb94a7dfa2', 'request_id': 'req-7371ecf9-45d7-4a3c-bbb9-4b3f93e00b37'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:23.044512 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1353/2704] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 23:02:22 2022] DELETE /identity/v3/projects/fe53d6ca4fa44a67ae4a039635d2c5b6 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 23:02:23.045779 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b063e6-ad18-4582-9af7-44ec00ff4d54 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:23.046065 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b063e6-ad18-4582-9af7-44ec00ff4d54 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:23.046342 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b063e6-ad18-4582-9af7-44ec00ff4d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:23.068566 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-53b063e6-ad18-4582-9af7-44ec00ff4d54 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:02:23.069887 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-53b063e6-ad18-4582-9af7-44ec00ff4d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:23.097508 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-7371ecf9-45d7-4a3c-bbb9-4b3f93e00b37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab5f235b30c54dedb41679fb94a7dfa2', 'request_id': 'req-7371ecf9-45d7-4a3c-bbb9-4b3f93e00b37'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:23.109638 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-7371ecf9-45d7-4a3c-bbb9-4b3f93e00b37 admin admin] The token cache is being invalidate because project ab5f235b30c54dedb41679fb94a7dfa2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:02:23.109841 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-7371ecf9-45d7-4a3c-bbb9-4b3f93e00b37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:23.110619 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-7371ecf9-45d7-4a3c-bbb9-4b3f93e00b37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:23.112208 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1352/2705] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 23:02:22 2022] DELETE /identity/v3/projects/ab5f235b30c54dedb41679fb94a7dfa2 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:23.115607 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7371ecf9-45d7-4a3c-bbb9-4b3f93e00b37 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:23.115607 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-7371ecf9-45d7-4a3c-bbb9-4b3f93e00b37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:23.168484 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1354/2706] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:02:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:02:23.170291 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e5fda-5ba7-4b1b-a627-58e62e57d8c5 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:23.170646 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e5fda-5ba7-4b1b-a627-58e62e57d8c5 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:23.171015 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e5fda-5ba7-4b1b-a627-58e62e57d8c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:23.190336 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-001e5fda-5ba7-4b1b-a627-58e62e57d8c5 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:02:23.203082 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-001e5fda-5ba7-4b1b-a627-58e62e57d8c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:23.203082 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1355/2707] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:02:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:02:23.208948 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324730cf-a6d7-4db2-b547-aec837bd315c None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:23.209244 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324730cf-a6d7-4db2-b547-aec837bd315c None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:23.209530 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324730cf-a6d7-4db2-b547-aec837bd315c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:23.217748 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7371ecf9-45d7-4a3c-bbb9-4b3f93e00b37 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:23.223010 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddb9a48-4c4a-4624-aa7c-d868ecedc5d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:23.224615 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb9a48-4c4a-4624-aa7c-d868ecedc5d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:23.224615 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb9a48-4c4a-4624-aa7c-d868ecedc5d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:23.224615 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb9a48-4c4a-4624-aa7c-d868ecedc5d3 admin admin] PATH_INFO: `/v3/projects/7cf2fec2943c4371844d205ae919d1ac` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:23.225853 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddb9a48-4c4a-4624-aa7c-d868ecedc5d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cf2fec2943c4371844d205ae919d1ac)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:23.228110 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddb9a48-4c4a-4624-aa7c-d868ecedc5d3 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:23.229608 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-324730cf-a6d7-4db2-b547-aec837bd315c None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:02:23.230907 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-324730cf-a6d7-4db2-b547-aec837bd315c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:23.239306 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1356/2708] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:02:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:02:23.241257 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a549cb0-b0e7-46e4-869c-27d5f390e843 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:23.241549 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a549cb0-b0e7-46e4-869c-27d5f390e843 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:23.241824 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a549cb0-b0e7-46e4-869c-27d5f390e843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:23.260874 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-5a549cb0-b0e7-46e4-869c-27d5f390e843 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:02:23.262045 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-5a549cb0-b0e7-46e4-869c-27d5f390e843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:23.270839 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1357/2709] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:02:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:02:23.276169 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54599544-7498-4727-be3c-189a8bce6427 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:23.276169 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54599544-7498-4727-be3c-189a8bce6427 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:23.276169 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54599544-7498-4727-be3c-189a8bce6427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:23.290496 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-54599544-7498-4727-be3c-189a8bce6427 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:02:23.291775 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-54599544-7498-4727-be3c-189a8bce6427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:23.299981 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1358/2710] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:02:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:02:23.311117 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4720207-46d0-4a88-ada8-10da487a3044 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:23.311117 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4720207-46d0-4a88-ada8-10da487a3044 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:23.311117 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4720207-46d0-4a88-ada8-10da487a3044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:23.320851 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-e4720207-46d0-4a88-ada8-10da487a3044 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:02:23.322080 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-e4720207-46d0-4a88-ada8-10da487a3044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:23.330558 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1359/2711] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:02:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:02:23.330558 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4720207-46d0-4a88-ada8-10da487a3044 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:23.341812 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4720207-46d0-4a88-ada8-10da487a3044 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:23.348183 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897da4d7-5ce1-4fef-bb9b-8ab30a7c8012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:23.348900 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897da4d7-5ce1-4fef-bb9b-8ab30a7c8012 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:23.349186 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897da4d7-5ce1-4fef-bb9b-8ab30a7c8012 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:23.349463 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897da4d7-5ce1-4fef-bb9b-8ab30a7c8012 admin admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:23.350404 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897da4d7-5ce1-4fef-bb9b-8ab30a7c8012 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 23:02:23.350715 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897da4d7-5ce1-4fef-bb9b-8ab30a7c8012 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:23.352394 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897da4d7-5ce1-4fef-bb9b-8ab30a7c8012 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:23.365831 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1360/2712] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 23:02:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 23:02:23.366554 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897da4d7-5ce1-4fef-bb9b-8ab30a7c8012 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:23.376949 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897da4d7-5ce1-4fef-bb9b-8ab30a7c8012 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:23.382279 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11011343-7d1e-4b7b-b8e1-479185b8cdad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:23.383002 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11011343-7d1e-4b7b-b8e1-479185b8cdad admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:23.383289 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11011343-7d1e-4b7b-b8e1-479185b8cdad admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:23.383566 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11011343-7d1e-4b7b-b8e1-479185b8cdad admin admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:23.384299 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11011343-7d1e-4b7b-b8e1-479185b8cdad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 23:02:23.384609 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11011343-7d1e-4b7b-b8e1-479185b8cdad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:23.386224 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11011343-7d1e-4b7b-b8e1-479185b8cdad admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:23.405837 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1361/2713] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 23:02:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 23:02:23.406874 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11011343-7d1e-4b7b-b8e1-479185b8cdad admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:23.419268 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11011343-7d1e-4b7b-b8e1-479185b8cdad admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:23.419895 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-4ddb9a48-4c4a-4624-aa7c-d868ecedc5d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cf2fec2943c4371844d205ae919d1ac', 'request_id': 'req-4ddb9a48-4c4a-4624-aa7c-d868ecedc5d3'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:23.426072 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-4ddb9a48-4c4a-4624-aa7c-d868ecedc5d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cf2fec2943c4371844d205ae919d1ac', 'request_id': 'req-4ddb9a48-4c4a-4624-aa7c-d868ecedc5d3'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:23.497091 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-4ddb9a48-4c4a-4624-aa7c-d868ecedc5d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cf2fec2943c4371844d205ae919d1ac', 'request_id': 'req-4ddb9a48-4c4a-4624-aa7c-d868ecedc5d3'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:23.500800 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb6144-6be3-4074-9c50-166950d9eaa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:23.501538 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb6144-6be3-4074-9c50-166950d9eaa1 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:23.501920 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb6144-6be3-4074-9c50-166950d9eaa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:23.502346 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb6144-6be3-4074-9c50-166950d9eaa1 admin admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:23.503427 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cb6144-6be3-4074-9c50-166950d9eaa1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 23:02:23.503845 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cb6144-6be3-4074-9c50-166950d9eaa1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:23.506224 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cb6144-6be3-4074-9c50-166950d9eaa1 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:23.509621 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:23.513088 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-4ddb9a48-4c4a-4624-aa7c-d868ecedc5d3 admin admin] The token cache is being invalidate because project 7cf2fec2943c4371844d205ae919d1ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:02:23.513711 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-4ddb9a48-4c4a-4624-aa7c-d868ecedc5d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:23.515108 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-4ddb9a48-4c4a-4624-aa7c-d868ecedc5d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:23.517529 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1353/2714] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 23:02:23 2022] DELETE /identity/v3/projects/7cf2fec2943c4371844d205ae919d1ac => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:23.518410 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddb9a48-4c4a-4624-aa7c-d868ecedc5d3 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:23.521658 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-4ddb9a48-4c4a-4624-aa7c-d868ecedc5d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:23.527095 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1362/2715] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 23:02:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 23:02:23.527095 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb6144-6be3-4074-9c50-166950d9eaa1 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:23.527095 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-24cb6144-6be3-4074-9c50-166950d9eaa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:23.539628 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddb9a48-4c4a-4624-aa7c-d868ecedc5d3 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:23.541049 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb6144-6be3-4074-9c50-166950d9eaa1 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:23.544873 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61340120-e740-44b4-b3fc-35b17750b55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:23.545946 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61340120-e740-44b4-b3fc-35b17750b55c admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:23.546218 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61340120-e740-44b4-b3fc-35b17750b55c admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:23.546516 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61340120-e740-44b4-b3fc-35b17750b55c admin admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:23.547251 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61340120-e740-44b4-b3fc-35b17750b55c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:23.550149 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61340120-e740-44b4-b3fc-35b17750b55c admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:23.555555 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a03c28-e9c2-4f7e-b9d1-372bc13568d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:23.556870 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a03c28-e9c2-4f7e-b9d1-372bc13568d6 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:23.557319 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a03c28-e9c2-4f7e-b9d1-372bc13568d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:23.557700 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a03c28-e9c2-4f7e-b9d1-372bc13568d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:23.558568 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a03c28-e9c2-4f7e-b9d1-372bc13568d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:23.561995 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a03c28-e9c2-4f7e-b9d1-372bc13568d6 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:23.656611 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1354/2716] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 23:02:23 2022] POST /identity/v3/projects => generated 399 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:02:23.656611 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afa0c07-0dae-4746-bf0b-2f44907462e9 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:23.656611 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afa0c07-0dae-4746-bf0b-2f44907462e9 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:23.656611 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afa0c07-0dae-4746-bf0b-2f44907462e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:23.676056 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-5afa0c07-0dae-4746-bf0b-2f44907462e9 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:02:23.677590 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:23.685670 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-5afa0c07-0dae-4746-bf0b-2f44907462e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:23.689764 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1363/2717] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 23:02:23 2022] POST /identity/v3/projects => generated 375 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:02:23.690733 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a03c28-e9c2-4f7e-b9d1-372bc13568d6 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:23.695421 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-71a03c28-e9c2-4f7e-b9d1-372bc13568d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:23.751436 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1355/2718] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:02:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:02:23.752552 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afa0c07-0dae-4746-bf0b-2f44907462e9 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:23.762310 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afa0c07-0dae-4746-bf0b-2f44907462e9 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:23.767175 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b8552d-b8df-48e4-ad6f-667c23844452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:23.767872 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8552d-b8df-48e4-ad6f-667c23844452 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:23.768164 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8552d-b8df-48e4-ad6f-667c23844452 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:23.768444 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8552d-b8df-48e4-ad6f-667c23844452 admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:23.769135 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b8552d-b8df-48e4-ad6f-667c23844452 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:23.771118 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b8552d-b8df-48e4-ad6f-667c23844452 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:23.772933 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a03c28-e9c2-4f7e-b9d1-372bc13568d6 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:23.778895 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a30017-4aea-45c3-91d0-ce6eef2cca63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:23.779571 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a30017-4aea-45c3-91d0-ce6eef2cca63 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:23.779862 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a30017-4aea-45c3-91d0-ce6eef2cca63 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:23.780196 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a30017-4aea-45c3-91d0-ce6eef2cca63 admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:23.780839 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a30017-4aea-45c3-91d0-ce6eef2cca63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:23.782807 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a30017-4aea-45c3-91d0-ce6eef2cca63 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:23.872573 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1356/2719] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 23:02:23 2022] POST /identity/v3/users => generated 300 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:02:23.876930 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b8552d-b8df-48e4-ad6f-667c23844452 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:23.883426 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b8552d-b8df-48e4-ad6f-667c23844452 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:23.888343 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a28fb3-8fdb-40dc-87f5-fe941acde183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:23.888998 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a28fb3-8fdb-40dc-87f5-fe941acde183 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:23.889283 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a28fb3-8fdb-40dc-87f5-fe941acde183 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:23.889562 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a28fb3-8fdb-40dc-87f5-fe941acde183 admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:23.890243 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a28fb3-8fdb-40dc-87f5-fe941acde183 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:23.892321 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a28fb3-8fdb-40dc-87f5-fe941acde183 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:23.912352 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1364/2720] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 23:02:23 2022] POST /identity/v3/projects => generated 365 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:02:23.912352 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137db465-6252-47e2-862f-8ed09412e408 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:23.912352 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137db465-6252-47e2-862f-8ed09412e408 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:23.912352 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137db465-6252-47e2-862f-8ed09412e408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:23.929893 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-137db465-6252-47e2-862f-8ed09412e408 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:02:23.931133 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-137db465-6252-47e2-862f-8ed09412e408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:23.989645 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1365/2721] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:02:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:02:23.990493 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137db465-6252-47e2-862f-8ed09412e408 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:23.999379 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137db465-6252-47e2-862f-8ed09412e408 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:24.003933 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9206f0d3-b93d-40c8-b11f-ae5337f35344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:24.011603 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206f0d3-b93d-40c8-b11f-ae5337f35344 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:24.011603 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206f0d3-b93d-40c8-b11f-ae5337f35344 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:24.011603 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206f0d3-b93d-40c8-b11f-ae5337f35344 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:24.011603 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9206f0d3-b93d-40c8-b11f-ae5337f35344 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:24.011603 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9206f0d3-b93d-40c8-b11f-ae5337f35344 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:24.017021 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1366/2722] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:02:23 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:02:24.017758 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9206f0d3-b93d-40c8-b11f-ae5337f35344 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:24.028230 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9206f0d3-b93d-40c8-b11f-ae5337f35344 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:24.032847 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fcdc38-d594-4031-bdde-5ad4296378fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:24.033418 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fcdc38-d594-4031-bdde-5ad4296378fb admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:24.033702 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fcdc38-d594-4031-bdde-5ad4296378fb admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:24.033984 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fcdc38-d594-4031-bdde-5ad4296378fb admin admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:24.034655 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fcdc38-d594-4031-bdde-5ad4296378fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:24.037209 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fcdc38-d594-4031-bdde-5ad4296378fb admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:24.041060 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1357/2723] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 23:02:23 2022] POST /identity/v3/users => generated 288 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:02:24.048360 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a28fb3-8fdb-40dc-87f5-fe941acde183 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:24.051886 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a28fb3-8fdb-40dc-87f5-fe941acde183 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:24.056709 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997b4564-668a-4355-8155-91cb74727405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:24.057352 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b4564-668a-4355-8155-91cb74727405 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:24.057660 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b4564-668a-4355-8155-91cb74727405 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:24.057942 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b4564-668a-4355-8155-91cb74727405 admin admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:24.058661 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997b4564-668a-4355-8155-91cb74727405 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 23:02:24.059004 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997b4564-668a-4355-8155-91cb74727405 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:24.060661 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997b4564-668a-4355-8155-91cb74727405 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:24.073454 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1358/2724] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 23:02:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 23:02:24.074369 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997b4564-668a-4355-8155-91cb74727405 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:24.084408 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997b4564-668a-4355-8155-91cb74727405 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:24.089077 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f442d2d-d235-4695-a793-2abcec0ca972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:24.089701 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f442d2d-d235-4695-a793-2abcec0ca972 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:24.089987 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f442d2d-d235-4695-a793-2abcec0ca972 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:24.090268 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f442d2d-d235-4695-a793-2abcec0ca972 admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:24.091040 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f442d2d-d235-4695-a793-2abcec0ca972 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:24.092575 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f442d2d-d235-4695-a793-2abcec0ca972 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:24.103123 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1359/2725] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:02:24.104077 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f442d2d-d235-4695-a793-2abcec0ca972 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:24.113650 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f442d2d-d235-4695-a793-2abcec0ca972 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:24.118364 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b0024-029e-4f44-b1a2-0df3ab224934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:24.119007 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b0024-029e-4f44-b1a2-0df3ab224934 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:24.119292 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b0024-029e-4f44-b1a2-0df3ab224934 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:24.119572 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b0024-029e-4f44-b1a2-0df3ab224934 admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:24.120286 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975b0024-029e-4f44-b1a2-0df3ab224934 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:24.121812 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975b0024-029e-4f44-b1a2-0df3ab224934 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:24.123404 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1367/2726] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 23:02:24 2022] POST /identity/v3/users => generated 283 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:02:24.128148 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fcdc38-d594-4031-bdde-5ad4296378fb admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:24.136234 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1360/2727] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:02:24.136408 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fcdc38-d594-4031-bdde-5ad4296378fb admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:24.136530 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b0024-029e-4f44-b1a2-0df3ab224934 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:24.140989 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb21a17-bd01-41d8-a520-543cf060075c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:24.141664 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb21a17-bd01-41d8-a520-543cf060075c admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:24.142200 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb21a17-bd01-41d8-a520-543cf060075c admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:24.142733 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb21a17-bd01-41d8-a520-543cf060075c admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:24.143846 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb21a17-bd01-41d8-a520-543cf060075c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:24.144306 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b0024-029e-4f44-b1a2-0df3ab224934 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:24.146570 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb21a17-bd01-41d8-a520-543cf060075c admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:24.149481 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f7e200-abb3-4808-910d-bad27b072a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:24.151742 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7e200-abb3-4808-910d-bad27b072a5f admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:24.151742 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7e200-abb3-4808-910d-bad27b072a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:24.151742 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7e200-abb3-4808-910d-bad27b072a5f admin admin] PATH_INFO: `/v3/projects/2b332ea671ec4c13b56113a2e7d30e31/users/b7cca344a5b842bbb9ff9a3595c18bd4/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:24.166062 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f7e200-abb3-4808-910d-bad27b072a5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b332ea671ec4c13b56113a2e7d30e31, user_id=b7cca344a5b842bbb9ff9a3595c18bd4, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:24.168699 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f7e200-abb3-4808-910d-bad27b072a5f admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:24.262228 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1368/2728] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 23:02:24 2022] POST /identity/v3/projects => generated 405 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:02:24.262228 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb21a17-bd01-41d8-a520-543cf060075c admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:24.281674 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:24.287563 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1361/2729] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 23:02:24 2022] PUT /identity/v3/projects/2b332ea671ec4c13b56113a2e7d30e31/users/b7cca344a5b842bbb9ff9a3595c18bd4/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:24.288610 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f7e200-abb3-4808-910d-bad27b072a5f admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:24.331903 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:DELETE=2,keystone:SELECT=89,keystone:INSERT=9 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:24.342873 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb21a17-bd01-41d8-a520-543cf060075c admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:24.370029 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f7e200-abb3-4808-910d-bad27b072a5f admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:24.374864 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d0eb2d-7f34-4702-92a5-1144ebf0806b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:24.375585 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0eb2d-7f34-4702-92a5-1144ebf0806b admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:24.375899 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0eb2d-7f34-4702-92a5-1144ebf0806b admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:24.376171 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0eb2d-7f34-4702-92a5-1144ebf0806b admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:24.376910 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d0eb2d-7f34-4702-92a5-1144ebf0806b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:24.378643 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d0eb2d-7f34-4702-92a5-1144ebf0806b admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:24.389556 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1362/2730] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:02:24.390342 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d0eb2d-7f34-4702-92a5-1144ebf0806b admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:24.400884 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d0eb2d-7f34-4702-92a5-1144ebf0806b admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:24.406080 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc680bbe-db41-49fd-92f8-ac6c3c5b59ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:24.406755 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc680bbe-db41-49fd-92f8-ac6c3c5b59ac admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:24.407069 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc680bbe-db41-49fd-92f8-ac6c3c5b59ac admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:24.407349 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc680bbe-db41-49fd-92f8-ac6c3c5b59ac admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:24.408024 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc680bbe-db41-49fd-92f8-ac6c3c5b59ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:24.408796 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b9d1cd-cc72-4dda-bf37-b34d127b8b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:24.409463 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b9d1cd-cc72-4dda-bf37-b34d127b8b4e admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:24.409748 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b9d1cd-cc72-4dda-bf37-b34d127b8b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:24.409954 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc680bbe-db41-49fd-92f8-ac6c3c5b59ac admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:24.410064 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b9d1cd-cc72-4dda-bf37-b34d127b8b4e admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:24.410814 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b9d1cd-cc72-4dda-bf37-b34d127b8b4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:24.412470 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b9d1cd-cc72-4dda-bf37-b34d127b8b4e admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:24.423084 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1369/2731] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:02:24.424165 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb351a5-5ba3-48c6-a46a-c68903296348 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:24.424453 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb351a5-5ba3-48c6-a46a-c68903296348 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:24.424732 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb351a5-5ba3-48c6-a46a-c68903296348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:24.450086 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-8cb351a5-5ba3-48c6-a46a-c68903296348 None None] MFA Rules not processed for user `b7cca344a5b842bbb9ff9a3595c18bd4`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:02:24.451315 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-8cb351a5-5ba3-48c6-a46a-c68903296348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:24.466433 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1363/2732] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 23:02:24 2022] POST /identity/v3/users => generated 303 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:02:24.466433 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc680bbe-db41-49fd-92f8-ac6c3c5b59ac admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:24.474959 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc680bbe-db41-49fd-92f8-ac6c3c5b59ac admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:24.482326 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:24.486812 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195069dd-d93b-435e-937b-e43c9c597afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:24.488007 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195069dd-d93b-435e-937b-e43c9c597afa admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:24.488298 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195069dd-d93b-435e-937b-e43c9c597afa admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:24.488577 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195069dd-d93b-435e-937b-e43c9c597afa admin admin] PATH_INFO: `/v3/projects/3dd7349d012948619faf1671b6637f1a/users/ccd0d42a04324e58a3cbca7c5729bdc5/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:24.511249 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195069dd-d93b-435e-937b-e43c9c597afa admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dd7349d012948619faf1671b6637f1a, user_id=ccd0d42a04324e58a3cbca7c5729bdc5, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:24.513947 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195069dd-d93b-435e-937b-e43c9c597afa admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:24.517192 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1370/2733] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:02:24 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:02:24.518991 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb351a5-5ba3-48c6-a46a-c68903296348 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:24.528647 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb351a5-5ba3-48c6-a46a-c68903296348 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:24.533294 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7a1843-d61f-4a12-8502-bde30670115a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:24.533947 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a1843-d61f-4a12-8502-bde30670115a admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:24.534228 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a1843-d61f-4a12-8502-bde30670115a admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:24.534503 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a1843-d61f-4a12-8502-bde30670115a admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:24.535268 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7a1843-d61f-4a12-8502-bde30670115a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:24.536873 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7a1843-d61f-4a12-8502-bde30670115a admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:24.546656 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1371/2734] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:02:24.549958 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7a1843-d61f-4a12-8502-bde30670115a admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:24.559805 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7a1843-d61f-4a12-8502-bde30670115a admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:24.564392 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9de5da8-e5fe-4787-8b78-383cd98f751e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:24.565255 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de5da8-e5fe-4787-8b78-383cd98f751e admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:24.565596 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de5da8-e5fe-4787-8b78-383cd98f751e admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:24.566033 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de5da8-e5fe-4787-8b78-383cd98f751e admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:24.566846 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9de5da8-e5fe-4787-8b78-383cd98f751e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:24.569278 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9de5da8-e5fe-4787-8b78-383cd98f751e admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:24.579425 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1372/2735] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:02:24.580198 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9de5da8-e5fe-4787-8b78-383cd98f751e admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:24.581844 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-d9de5da8-e5fe-4787-8b78-383cd98f751e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:24.595690 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1364/2736] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 23:02:24 2022] PUT /identity/v3/projects/3dd7349d012948619faf1671b6637f1a/users/ccd0d42a04324e58a3cbca7c5729bdc5/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:24.596765 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195069dd-d93b-435e-937b-e43c9c597afa admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:24.644043 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9de5da8-e5fe-4787-8b78-383cd98f751e admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:24.666605 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195069dd-d93b-435e-937b-e43c9c597afa admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:24.671400 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a333f-9817-4c96-8b21-0ffc9b165088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:24.672297 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a333f-9817-4c96-8b21-0ffc9b165088 admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:24.672587 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a333f-9817-4c96-8b21-0ffc9b165088 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:24.672872 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a333f-9817-4c96-8b21-0ffc9b165088 admin admin] PATH_INFO: `/v3/projects/935079a6a1634e61a15eb5740c3cdfb9/users/238eb2d2b5c44d419a5d41267bd4dce0/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:24.687593 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a333f-9817-4c96-8b21-0ffc9b165088 admin admin] RBAC: Authorizing `identity:create_grant(project_id=935079a6a1634e61a15eb5740c3cdfb9, user_id=238eb2d2b5c44d419a5d41267bd4dce0, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:24.690164 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a333f-9817-4c96-8b21-0ffc9b165088 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:24.693756 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b3088f-bd7d-4408-a214-443f74b2c1dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:24.694416 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b3088f-bd7d-4408-a214-443f74b2c1dd service cinder] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:24.694699 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b3088f-bd7d-4408-a214-443f74b2c1dd service cinder] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:24.695017 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b3088f-bd7d-4408-a214-443f74b2c1dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:24.751539 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b3088f-bd7d-4408-a214-443f74b2c1dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:24.752978 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b3088f-bd7d-4408-a214-443f74b2c1dd service cinder] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:24.762587 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1373/2737] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:02:24 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:02:24.763495 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b3088f-bd7d-4408-a214-443f74b2c1dd service cinder] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:24.773693 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b3088f-bd7d-4408-a214-443f74b2c1dd service cinder] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:24.778354 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89151c59-8810-4f83-90bd-7a0bf6b73a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:24.779101 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89151c59-8810-4f83-90bd-7a0bf6b73a3c admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:24.779383 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89151c59-8810-4f83-90bd-7a0bf6b73a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:24.779661 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89151c59-8810-4f83-90bd-7a0bf6b73a3c admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:24.780363 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89151c59-8810-4f83-90bd-7a0bf6b73a3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:24.785929 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89151c59-8810-4f83-90bd-7a0bf6b73a3c admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:24.792518 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1374/2738] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:02:24.793766 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc49441-1ea6-4ac0-bf1b-e250c95066b9 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:24.794049 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc49441-1ea6-4ac0-bf1b-e250c95066b9 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:24.794446 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc49441-1ea6-4ac0-bf1b-e250c95066b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:24.799183 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1365/2739] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 23:02:24 2022] PUT /identity/v3/projects/935079a6a1634e61a15eb5740c3cdfb9/users/238eb2d2b5c44d419a5d41267bd4dce0/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:24.801614 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a333f-9817-4c96-8b21-0ffc9b165088 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:24.808651 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=100 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:24.824598 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-9fc49441-1ea6-4ac0-bf1b-e250c95066b9 None None] MFA Rules not processed for user `ccd0d42a04324e58a3cbca7c5729bdc5`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:02:24.826367 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-9fc49441-1ea6-4ac0-bf1b-e250c95066b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:24.896929 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a333f-9817-4c96-8b21-0ffc9b165088 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:24.898074 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1375/2740] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:02:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:02:24.899401 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1e861-4856-4bdd-b93e-630d7c548bc2 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:24.899672 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1e861-4856-4bdd-b93e-630d7c548bc2 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:24.899944 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1e861-4856-4bdd-b93e-630d7c548bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:24.909508 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedbcbe7-e950-44e4-a6a3-51b451b34c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:24.910260 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedbcbe7-e950-44e4-a6a3-51b451b34c07 admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:24.910537 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedbcbe7-e950-44e4-a6a3-51b451b34c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:24.910845 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedbcbe7-e950-44e4-a6a3-51b451b34c07 admin admin] PATH_INFO: `/v3/projects/37ff6f2a7d5f4ef5bb8721dbda85a2d3/users/96389402d7c64fd5b51dea90d71619b1/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:24.919345 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:24.926470 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-41b1e861-4856-4bdd-b93e-630d7c548bc2 None None] MFA Rules not processed for user `238eb2d2b5c44d419a5d41267bd4dce0`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:02:24.926717 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedbcbe7-e950-44e4-a6a3-51b451b34c07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37ff6f2a7d5f4ef5bb8721dbda85a2d3, user_id=96389402d7c64fd5b51dea90d71619b1, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:24.928225 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-41b1e861-4856-4bdd-b93e-630d7c548bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:24.930212 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedbcbe7-e950-44e4-a6a3-51b451b34c07 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:24.992833 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1376/2741] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:02:24 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:02:24.993761 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b1e861-4856-4bdd-b93e-630d7c548bc2 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:25.004770 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b1e861-4856-4bdd-b93e-630d7c548bc2 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:25.009445 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec58ba63-d0e0-43a0-9930-d665981f6352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:25.010095 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec58ba63-d0e0-43a0-9930-d665981f6352 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:25.010406 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec58ba63-d0e0-43a0-9930-d665981f6352 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:25.010680 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec58ba63-d0e0-43a0-9930-d665981f6352 admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:25.011386 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec58ba63-d0e0-43a0-9930-d665981f6352 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:25.013301 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec58ba63-d0e0-43a0-9930-d665981f6352 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:25.023196 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1366/2742] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 23:02:24 2022] PUT /identity/v3/projects/37ff6f2a7d5f4ef5bb8721dbda85a2d3/users/96389402d7c64fd5b51dea90d71619b1/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:25.024290 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedbcbe7-e950-44e4-a6a3-51b451b34c07 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:25.081263 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedbcbe7-e950-44e4-a6a3-51b451b34c07 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:25.085026 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a2e50b-8b37-43ea-8619-98a569c4ea1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:25.085651 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a2e50b-8b37-43ea-8619-98a569c4ea1f service cinder] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:25.085940 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a2e50b-8b37-43ea-8619-98a569c4ea1f service cinder] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:25.086218 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a2e50b-8b37-43ea-8619-98a569c4ea1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:25.097435 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1377/2743] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 23:02:24 2022] POST /identity/v3/projects => generated 377 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:02:25.099423 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33914f2-9681-4829-9faa-edc0b5522c94 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:25.099718 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33914f2-9681-4829-9faa-edc0b5522c94 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:25.099999 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33914f2-9681-4829-9faa-edc0b5522c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:25.127361 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-d33914f2-9681-4829-9faa-edc0b5522c94 None None] MFA Rules not processed for user `96389402d7c64fd5b51dea90d71619b1`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:02:25.128607 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-d33914f2-9681-4829-9faa-edc0b5522c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:25.152280 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a2e50b-8b37-43ea-8619-98a569c4ea1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:25.153708 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a2e50b-8b37-43ea-8619-98a569c4ea1f service cinder] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:25.163151 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1367/2744] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:02:25 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:02:25.163924 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a2e50b-8b37-43ea-8619-98a569c4ea1f service cinder] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:25.195294 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:25.208024 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1378/2745] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:02:25 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:02:25.212386 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33914f2-9681-4829-9faa-edc0b5522c94 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:25.248075 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a2e50b-8b37-43ea-8619-98a569c4ea1f service cinder] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:25.280916 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33914f2-9681-4829-9faa-edc0b5522c94 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:25.286002 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4afe7d-f006-45a1-9d2e-1815d26f5e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:25.287375 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4afe7d-f006-45a1-9d2e-1815d26f5e60 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:25.287712 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4afe7d-f006-45a1-9d2e-1815d26f5e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:25.287990 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4afe7d-f006-45a1-9d2e-1815d26f5e60 admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:25.288723 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4afe7d-f006-45a1-9d2e-1815d26f5e60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:25.291071 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4afe7d-f006-45a1-9d2e-1815d26f5e60 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:25.305057 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:25.317859 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88bf1bf-e0bf-48a2-b31b-7f4a66f40163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:25.318536 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88bf1bf-e0bf-48a2-b31b-7f4a66f40163 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:25.318850 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88bf1bf-e0bf-48a2-b31b-7f4a66f40163 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:25.319129 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88bf1bf-e0bf-48a2-b31b-7f4a66f40163 admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:25.319819 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88bf1bf-e0bf-48a2-b31b-7f4a66f40163 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:25.321743 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88bf1bf-e0bf-48a2-b31b-7f4a66f40163 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:25.387880 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1379/2746] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 23:02:25 2022] POST /identity/v3/projects => generated 405 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 23:02:25.391930 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4afe7d-f006-45a1-9d2e-1815d26f5e60 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:25.452308 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1368/2747] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 23:02:25 2022] POST /identity/v3/users => generated 289 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:02:25.453071 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4afe7d-f006-45a1-9d2e-1815d26f5e60 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:25.453210 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88bf1bf-e0bf-48a2-b31b-7f4a66f40163 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:25.457771 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efad374-9412-4fb0-b292-38c45c97e023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:25.458965 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efad374-9412-4fb0-b292-38c45c97e023 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:25.459253 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efad374-9412-4fb0-b292-38c45c97e023 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:25.459538 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efad374-9412-4fb0-b292-38c45c97e023 admin admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:25.460234 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efad374-9412-4fb0-b292-38c45c97e023 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:25.462158 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efad374-9412-4fb0-b292-38c45c97e023 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:25.463242 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88bf1bf-e0bf-48a2-b31b-7f4a66f40163 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:25.468091 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ee227b-9942-43d1-9fff-12616913d9f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:25.469090 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee227b-9942-43d1-9fff-12616913d9f7 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:25.469090 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee227b-9942-43d1-9fff-12616913d9f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:25.469290 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee227b-9942-43d1-9fff-12616913d9f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:25.470095 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ee227b-9942-43d1-9fff-12616913d9f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:25.471614 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ee227b-9942-43d1-9fff-12616913d9f7 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:25.485627 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1369/2748] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:02:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 23:02:25.489948 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ee227b-9942-43d1-9fff-12616913d9f7 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:25.500035 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ee227b-9942-43d1-9fff-12616913d9f7 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:25.505011 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fe47fc-f1c0-4620-930c-117c93b2124c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:25.505384 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe47fc-f1c0-4620-930c-117c93b2124c admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:25.505667 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe47fc-f1c0-4620-930c-117c93b2124c admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:25.505940 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe47fc-f1c0-4620-930c-117c93b2124c admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:25.506664 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fe47fc-f1c0-4620-930c-117c93b2124c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:25.508270 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fe47fc-f1c0-4620-930c-117c93b2124c admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:25.519014 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1370/2749] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:02:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 23:02:25.523555 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fe47fc-f1c0-4620-930c-117c93b2124c admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:25.533109 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fe47fc-f1c0-4620-930c-117c93b2124c admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:25.538286 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600ebfc7-116a-4bb2-9913-39026da78f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:25.539324 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ebfc7-116a-4bb2-9913-39026da78f43 admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:25.539605 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ebfc7-116a-4bb2-9913-39026da78f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:25.539877 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ebfc7-116a-4bb2-9913-39026da78f43 admin admin] PATH_INFO: `/v3/projects/4021f07357d24ea99e72dce9e94281b2/users/7a9e9c34faef44d3a9e7303d7494f2d7/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:25.559774 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600ebfc7-116a-4bb2-9913-39026da78f43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4021f07357d24ea99e72dce9e94281b2, user_id=7a9e9c34faef44d3a9e7303d7494f2d7, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:25.569590 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600ebfc7-116a-4bb2-9913-39026da78f43 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:25.569727 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1380/2750] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 23:02:25 2022] POST /identity/v3/users => generated 309 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 23:02:25.569727 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efad374-9412-4fb0-b292-38c45c97e023 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:25.576096 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efad374-9412-4fb0-b292-38c45c97e023 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:25.582813 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c6f864-b5ab-4e2f-8abf-a8658afdebce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:25.583132 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6f864-b5ab-4e2f-8abf-a8658afdebce admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:25.584003 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6f864-b5ab-4e2f-8abf-a8658afdebce admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:25.584346 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6f864-b5ab-4e2f-8abf-a8658afdebce admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:25.585149 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c6f864-b5ab-4e2f-8abf-a8658afdebce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:25.586837 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c6f864-b5ab-4e2f-8abf-a8658afdebce admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:25.597452 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1381/2751] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:02:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:02:25.603796 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c6f864-b5ab-4e2f-8abf-a8658afdebce admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:25.613771 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c6f864-b5ab-4e2f-8abf-a8658afdebce admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:25.618367 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86165816-fabe-417a-89b8-59e26f7c898c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:25.619971 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86165816-fabe-417a-89b8-59e26f7c898c admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:25.620412 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86165816-fabe-417a-89b8-59e26f7c898c admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:25.620686 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86165816-fabe-417a-89b8-59e26f7c898c admin admin] PATH_INFO: `/v3/projects/f201595a8c9f4b9289ac4585a753e828/users/586f35be84224aa993129dc4f9a3e196/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:25.628982 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1371/2752] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 23:02:25 2022] PUT /identity/v3/projects/4021f07357d24ea99e72dce9e94281b2/users/7a9e9c34faef44d3a9e7303d7494f2d7/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 23:02:25.639886 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86165816-fabe-417a-89b8-59e26f7c898c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f201595a8c9f4b9289ac4585a753e828, user_id=586f35be84224aa993129dc4f9a3e196, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:25.640476 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba0bbec-9051-49eb-843b-8a5348317e65 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:25.640837 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba0bbec-9051-49eb-843b-8a5348317e65 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:25.641185 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba0bbec-9051-49eb-843b-8a5348317e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:25.642761 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86165816-fabe-417a-89b8-59e26f7c898c admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:25.672130 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-2ba0bbec-9051-49eb-843b-8a5348317e65 None None] MFA Rules not processed for user `7a9e9c34faef44d3a9e7303d7494f2d7`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:02:25.673454 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-2ba0bbec-9051-49eb-843b-8a5348317e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:25.715560 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1382/2753] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 23:02:25 2022] PUT /identity/v3/projects/f201595a8c9f4b9289ac4585a753e828/users/586f35be84224aa993129dc4f9a3e196/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:25.719134 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86165816-fabe-417a-89b8-59e26f7c898c admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:25.739728 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1372/2754] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:02:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 23:02:25.746110 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba0bbec-9051-49eb-843b-8a5348317e65 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:25.755370 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:25.786000 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86165816-fabe-417a-89b8-59e26f7c898c admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:25.793042 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cebe3f-4cdb-4c6f-bcfc-5afc8aa34143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:25.800928 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cebe3f-4cdb-4c6f-bcfc-5afc8aa34143 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:25.800928 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cebe3f-4cdb-4c6f-bcfc-5afc8aa34143 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:25.800928 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cebe3f-4cdb-4c6f-bcfc-5afc8aa34143 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:25.800928 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cebe3f-4cdb-4c6f-bcfc-5afc8aa34143 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:25.800928 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cebe3f-4cdb-4c6f-bcfc-5afc8aa34143 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:25.807528 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1383/2755] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:02:25 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:02:25.811551 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cebe3f-4cdb-4c6f-bcfc-5afc8aa34143 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:25.814734 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba0bbec-9051-49eb-843b-8a5348317e65 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:25.819716 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfc1d0b-32d6-4f73-8ed9-d949319136a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:25.820265 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc1d0b-32d6-4f73-8ed9-d949319136a4 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:25.820551 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc1d0b-32d6-4f73-8ed9-d949319136a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:25.820830 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc1d0b-32d6-4f73-8ed9-d949319136a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:25.821408 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cebe3f-4cdb-4c6f-bcfc-5afc8aa34143 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:25.821537 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfc1d0b-32d6-4f73-8ed9-d949319136a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:25.823462 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfc1d0b-32d6-4f73-8ed9-d949319136a4 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:25.826060 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bb278a-3281-47b1-b3cc-9f6644ea7bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:25.826966 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb278a-3281-47b1-b3cc-9f6644ea7bf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:25.827251 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb278a-3281-47b1-b3cc-9f6644ea7bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:25.827529 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb278a-3281-47b1-b3cc-9f6644ea7bf6 admin admin] PATH_INFO: `/v3/projects/f201595a8c9f4b9289ac4585a753e828/users/586f35be84224aa993129dc4f9a3e196/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:25.829526 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bb278a-3281-47b1-b3cc-9f6644ea7bf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f201595a8c9f4b9289ac4585a753e828, user_id=586f35be84224aa993129dc4f9a3e196, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:25.832227 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bb278a-3281-47b1-b3cc-9f6644ea7bf6 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:25.843312 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1384/2756] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 23:02:25 2022] PUT /identity/v3/projects/f201595a8c9f4b9289ac4585a753e828/users/586f35be84224aa993129dc4f9a3e196/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 23:02:25.854171 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41845c0f-f091-4c5a-86c5-f15b7b8a1ede None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:25.854727 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41845c0f-f091-4c5a-86c5-f15b7b8a1ede None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:25.855090 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41845c0f-f091-4c5a-86c5-f15b7b8a1ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:25.882179 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-41845c0f-f091-4c5a-86c5-f15b7b8a1ede None None] MFA Rules not processed for user `586f35be84224aa993129dc4f9a3e196`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:02:25.883432 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-41845c0f-f091-4c5a-86c5-f15b7b8a1ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:25.900710 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1373/2757] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 23:02:25 2022] POST /identity/v3/projects => generated 375 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 23:02:25.904421 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfc1d0b-32d6-4f73-8ed9-d949319136a4 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:25.913805 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:25.968860 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1385/2758] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:02:25 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 23:02:25.974419 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfc1d0b-32d6-4f73-8ed9-d949319136a4 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:25.977233 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41845c0f-f091-4c5a-86c5-f15b7b8a1ede None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:25.981780 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca5d75b-038f-4f1c-ad28-d92adedda193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:25.982432 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5d75b-038f-4f1c-ad28-d92adedda193 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:25.982720 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5d75b-038f-4f1c-ad28-d92adedda193 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:25.983111 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5d75b-038f-4f1c-ad28-d92adedda193 admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:25.983872 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca5d75b-038f-4f1c-ad28-d92adedda193 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:25.986144 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca5d75b-038f-4f1c-ad28-d92adedda193 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:26.034825 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41845c0f-f091-4c5a-86c5-f15b7b8a1ede None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:26.082052 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736ac418-7e0f-4c0c-bba5-e57a851934f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:26.082699 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ac418-7e0f-4c0c-bba5-e57a851934f1 service cinder] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:26.083016 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ac418-7e0f-4c0c-bba5-e57a851934f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:26.083861 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ac418-7e0f-4c0c-bba5-e57a851934f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:26.093260 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:26.095682 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1374/2759] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 23:02:25 2022] POST /identity/v3/users => generated 294 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 23:02:26.100124 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca5d75b-038f-4f1c-ad28-d92adedda193 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:26.109759 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca5d75b-038f-4f1c-ad28-d92adedda193 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:26.114406 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fa6ac7-a785-4ac2-b11d-0cc879903487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:26.115163 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa6ac7-a785-4ac2-b11d-0cc879903487 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:26.115446 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa6ac7-a785-4ac2-b11d-0cc879903487 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:26.115725 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa6ac7-a785-4ac2-b11d-0cc879903487 admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:26.116460 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fa6ac7-a785-4ac2-b11d-0cc879903487 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:26.118088 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fa6ac7-a785-4ac2-b11d-0cc879903487 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:26.128386 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1375/2760] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:02:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 23:02:26.133065 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fa6ac7-a785-4ac2-b11d-0cc879903487 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:26.143220 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fa6ac7-a785-4ac2-b11d-0cc879903487 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:26.144822 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736ac418-7e0f-4c0c-bba5-e57a851934f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:26.147001 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736ac418-7e0f-4c0c-bba5-e57a851934f1 service cinder] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:26.148913 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a18ec71-6959-419c-9d3d-62aeeaa6dd4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:26.149822 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18ec71-6959-419c-9d3d-62aeeaa6dd4e admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:26.150101 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18ec71-6959-419c-9d3d-62aeeaa6dd4e admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:26.150383 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18ec71-6959-419c-9d3d-62aeeaa6dd4e admin admin] PATH_INFO: `/v3/projects/eaee93ac6ae3461da819e4cf729cf9b7/users/ab57ec5020a74ba0af0dc4c03ff89568/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:26.157273 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1386/2761] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:02:25 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 23:02:26.167194 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a18ec71-6959-419c-9d3d-62aeeaa6dd4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaee93ac6ae3461da819e4cf729cf9b7, user_id=ab57ec5020a74ba0af0dc4c03ff89568, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:26.170596 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a18ec71-6959-419c-9d3d-62aeeaa6dd4e admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:26.267285 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1376/2762] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 23:02:26 2022] PUT /identity/v3/projects/eaee93ac6ae3461da819e4cf729cf9b7/users/ab57ec5020a74ba0af0dc4c03ff89568/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:26.268068 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736ac418-7e0f-4c0c-bba5-e57a851934f1 service cinder] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:26.278744 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736ac418-7e0f-4c0c-bba5-e57a851934f1 service cinder] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:26.297494 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a18ec71-6959-419c-9d3d-62aeeaa6dd4e admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:26.340042 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecef15a-0ef3-42b5-a205-5246c7b5ff6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:26.340712 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecef15a-0ef3-42b5-a205-5246c7b5ff6a admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:26.341055 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecef15a-0ef3-42b5-a205-5246c7b5ff6a admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:26.341331 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecef15a-0ef3-42b5-a205-5246c7b5ff6a admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:26.342048 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecef15a-0ef3-42b5-a205-5246c7b5ff6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:26.343679 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecef15a-0ef3-42b5-a205-5246c7b5ff6a admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:26.354668 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1387/2763] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:02:26 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 23:02:26.358834 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecef15a-0ef3-42b5-a205-5246c7b5ff6a admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:26.363431 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a18ec71-6959-419c-9d3d-62aeeaa6dd4e admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:26.368680 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ce7c21-684b-474a-acd8-1df4152aa455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:26.369094 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecef15a-0ef3-42b5-a205-5246c7b5ff6a admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:26.369552 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ce7c21-684b-474a-acd8-1df4152aa455 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:26.369838 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ce7c21-684b-474a-acd8-1df4152aa455 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:26.370115 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ce7c21-684b-474a-acd8-1df4152aa455 admin admin] PATH_INFO: `/v3/users/96389402d7c64fd5b51dea90d71619b1` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:26.371405 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ce7c21-684b-474a-acd8-1df4152aa455 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96389402d7c64fd5b51dea90d71619b1)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:26.373265 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ce7c21-684b-474a-acd8-1df4152aa455 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:26.374069 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c6403-6a6f-4ee4-815d-7699c191b17e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:26.374956 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c6403-6a6f-4ee4-815d-7699c191b17e admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:26.375307 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c6403-6a6f-4ee4-815d-7699c191b17e admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:26.375584 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c6403-6a6f-4ee4-815d-7699c191b17e admin admin] PATH_INFO: `/v3/projects/eaee93ac6ae3461da819e4cf729cf9b7/users/ab57ec5020a74ba0af0dc4c03ff89568/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:26.377838 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739c6403-6a6f-4ee4-815d-7699c191b17e admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaee93ac6ae3461da819e4cf729cf9b7, user_id=ab57ec5020a74ba0af0dc4c03ff89568, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:26.381497 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739c6403-6a6f-4ee4-815d-7699c191b17e admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:26.392274 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1388/2764] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 23:02:26 2022] PUT /identity/v3/projects/eaee93ac6ae3461da819e4cf729cf9b7/users/ab57ec5020a74ba0af0dc4c03ff89568/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 23:02:26.402704 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be6b50-e36a-4f64-90ae-afbef8a215f3 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:26.403097 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be6b50-e36a-4f64-90ae-afbef8a215f3 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:26.403378 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be6b50-e36a-4f64-90ae-afbef8a215f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:26.430153 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-31be6b50-e36a-4f64-90ae-afbef8a215f3 None None] MFA Rules not processed for user `ab57ec5020a74ba0af0dc4c03ff89568`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:02:26.431381 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-31be6b50-e36a-4f64-90ae-afbef8a215f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:26.499919 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1389/2765] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:02:26 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 23:02:26.512361 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31be6b50-e36a-4f64-90ae-afbef8a215f3 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:26.530230 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:26.566827 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31be6b50-e36a-4f64-90ae-afbef8a215f3 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:26.571277 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3728eae7-46f1-45b0-a9c0-2a03a1b89204 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:26.571924 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3728eae7-46f1-45b0-a9c0-2a03a1b89204 service cinder] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:26.572209 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3728eae7-46f1-45b0-a9c0-2a03a1b89204 service cinder] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:26.572486 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3728eae7-46f1-45b0-a9c0-2a03a1b89204 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:26.574880 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-92ce7c21-684b-474a-acd8-1df4152aa455 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96389402d7c64fd5b51dea90d71619b1'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:26.582817 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-92ce7c21-684b-474a-acd8-1df4152aa455 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96389402d7c64fd5b51dea90d71619b1'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:26.583622 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-92ce7c21-684b-474a-acd8-1df4152aa455 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96389402d7c64fd5b51dea90d71619b1'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:26.589872 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:26.597936 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-92ce7c21-684b-474a-acd8-1df4152aa455 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96389402d7c64fd5b51dea90d71619b1'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:26.598690 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-92ce7c21-684b-474a-acd8-1df4152aa455 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96389402d7c64fd5b51dea90d71619b1'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:26.638982 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3728eae7-46f1-45b0-a9c0-2a03a1b89204 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:26.640554 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3728eae7-46f1-45b0-a9c0-2a03a1b89204 service cinder] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:26.649937 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1390/2766] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:02:26 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 23:02:26.671920 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1377/2767] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 23:02:26 2022] DELETE /identity/v3/users/96389402d7c64fd5b51dea90d71619b1 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 23:02:26.678753 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf545c4-f6c7-4570-81ce-842a9bf879e6 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:26.679064 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf545c4-f6c7-4570-81ce-842a9bf879e6 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:26.679340 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf545c4-f6c7-4570-81ce-842a9bf879e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:26.698424 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-fcf545c4-f6c7-4570-81ce-842a9bf879e6 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:02:26.700457 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-fcf545c4-f6c7-4570-81ce-842a9bf879e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:26.766472 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1391/2768] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:02:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:02:26.776109 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ce7c21-684b-474a-acd8-1df4152aa455 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:26.777595 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-92ce7c21-684b-474a-acd8-1df4152aa455 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:26.839636 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ce7c21-684b-474a-acd8-1df4152aa455 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:26.843311 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b996395-05a3-4b68-829e-3e91e202bae3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:26.843957 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b996395-05a3-4b68-829e-3e91e202bae3 service neutron] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:26.849330 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b996395-05a3-4b68-829e-3e91e202bae3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:26.849330 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b996395-05a3-4b68-829e-3e91e202bae3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:26.854560 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b996395-05a3-4b68-829e-3e91e202bae3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:26.856111 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b996395-05a3-4b68-829e-3e91e202bae3 service neutron] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:26.866248 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1378/2769] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:02:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:02:27.382056 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf545c4-f6c7-4570-81ce-842a9bf879e6 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:27.383501 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-fcf545c4-f6c7-4570-81ce-842a9bf879e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:27.396380 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf545c4-f6c7-4570-81ce-842a9bf879e6 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:27.401487 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352f67ae-5cd8-4198-b8f6-cc48bff52bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:27.402374 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352f67ae-5cd8-4198-b8f6-cc48bff52bca admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:27.402639 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352f67ae-5cd8-4198-b8f6-cc48bff52bca admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:27.402967 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352f67ae-5cd8-4198-b8f6-cc48bff52bca admin admin] PATH_INFO: `/v3/projects/37ff6f2a7d5f4ef5bb8721dbda85a2d3` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:27.404224 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352f67ae-5cd8-4198-b8f6-cc48bff52bca admin admin] RBAC: Authorizing `identity:delete_project(project_id=37ff6f2a7d5f4ef5bb8721dbda85a2d3)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:27.406073 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352f67ae-5cd8-4198-b8f6-cc48bff52bca admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:27.548650 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-352f67ae-5cd8-4198-b8f6-cc48bff52bca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37ff6f2a7d5f4ef5bb8721dbda85a2d3', 'request_id': 'req-352f67ae-5cd8-4198-b8f6-cc48bff52bca'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:27.555582 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-352f67ae-5cd8-4198-b8f6-cc48bff52bca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37ff6f2a7d5f4ef5bb8721dbda85a2d3', 'request_id': 'req-352f67ae-5cd8-4198-b8f6-cc48bff52bca'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:27.673919 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-352f67ae-5cd8-4198-b8f6-cc48bff52bca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37ff6f2a7d5f4ef5bb8721dbda85a2d3', 'request_id': 'req-352f67ae-5cd8-4198-b8f6-cc48bff52bca'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:27.675366 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:27.686246 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-352f67ae-5cd8-4198-b8f6-cc48bff52bca admin admin] The token cache is being invalidate because project 37ff6f2a7d5f4ef5bb8721dbda85a2d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:02:27.686536 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-352f67ae-5cd8-4198-b8f6-cc48bff52bca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:27.687214 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-352f67ae-5cd8-4198-b8f6-cc48bff52bca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:27.688771 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1392/2770] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 23:02:27 2022] DELETE /identity/v3/projects/37ff6f2a7d5f4ef5bb8721dbda85a2d3 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:27.693149 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b996395-05a3-4b68-829e-3e91e202bae3 service neutron] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:27.702687 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b996395-05a3-4b68-829e-3e91e202bae3 service neutron] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:27.764464 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a1eee0-679d-401f-83e7-7f6b42dfcfc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:27.765337 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1eee0-679d-401f-83e7-7f6b42dfcfc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:27.765619 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1eee0-679d-401f-83e7-7f6b42dfcfc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:27.765892 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1eee0-679d-401f-83e7-7f6b42dfcfc3 admin admin] PATH_INFO: `/v3/users/586f35be84224aa993129dc4f9a3e196` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:27.767095 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a1eee0-679d-401f-83e7-7f6b42dfcfc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=586f35be84224aa993129dc4f9a3e196)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:27.768990 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a1eee0-679d-401f-83e7-7f6b42dfcfc3 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:27.939715 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-99a1eee0-679d-401f-83e7-7f6b42dfcfc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '586f35be84224aa993129dc4f9a3e196'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:27.947487 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-99a1eee0-679d-401f-83e7-7f6b42dfcfc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '586f35be84224aa993129dc4f9a3e196'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:27.948169 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-99a1eee0-679d-401f-83e7-7f6b42dfcfc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '586f35be84224aa993129dc4f9a3e196'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:27.960547 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-99a1eee0-679d-401f-83e7-7f6b42dfcfc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '586f35be84224aa993129dc4f9a3e196'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:27.961180 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-99a1eee0-679d-401f-83e7-7f6b42dfcfc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '586f35be84224aa993129dc4f9a3e196'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:28.068386 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1379/2771] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 23:02:27 2022] DELETE /identity/v3/users/586f35be84224aa993129dc4f9a3e196 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:28.528880 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352f67ae-5cd8-4198-b8f6-cc48bff52bca admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:28.530490 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-352f67ae-5cd8-4198-b8f6-cc48bff52bca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:28.550101 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352f67ae-5cd8-4198-b8f6-cc48bff52bca admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:28.609201 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f0ef1-75e5-4b25-97b2-9cc7af4843a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:28.610137 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f0ef1-75e5-4b25-97b2-9cc7af4843a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:28.610417 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f0ef1-75e5-4b25-97b2-9cc7af4843a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:28.610693 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f0ef1-75e5-4b25-97b2-9cc7af4843a2 admin admin] PATH_INFO: `/v3/projects/f201595a8c9f4b9289ac4585a753e828` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:28.611895 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7f0ef1-75e5-4b25-97b2-9cc7af4843a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f201595a8c9f4b9289ac4585a753e828)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:28.613815 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7f0ef1-75e5-4b25-97b2-9cc7af4843a2 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:28.712234 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-4e7f0ef1-75e5-4b25-97b2-9cc7af4843a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f201595a8c9f4b9289ac4585a753e828', 'request_id': 'req-4e7f0ef1-75e5-4b25-97b2-9cc7af4843a2'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:28.717769 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-4e7f0ef1-75e5-4b25-97b2-9cc7af4843a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f201595a8c9f4b9289ac4585a753e828', 'request_id': 'req-4e7f0ef1-75e5-4b25-97b2-9cc7af4843a2'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:28.780391 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-4e7f0ef1-75e5-4b25-97b2-9cc7af4843a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f201595a8c9f4b9289ac4585a753e828', 'request_id': 'req-4e7f0ef1-75e5-4b25-97b2-9cc7af4843a2'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:28.791939 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-4e7f0ef1-75e5-4b25-97b2-9cc7af4843a2 admin admin] The token cache is being invalidate because project f201595a8c9f4b9289ac4585a753e828 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:02:28.797311 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-4e7f0ef1-75e5-4b25-97b2-9cc7af4843a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:28.797311 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-4e7f0ef1-75e5-4b25-97b2-9cc7af4843a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:28.797311 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1393/2772] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 23:02:28 2022] DELETE /identity/v3/projects/f201595a8c9f4b9289ac4585a753e828 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:28.799454 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d535fe-f046-41f2-b6bd-6c0ec1528f04 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:28.799751 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d535fe-f046-41f2-b6bd-6c0ec1528f04 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:28.800033 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d535fe-f046-41f2-b6bd-6c0ec1528f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:28.806423 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:28.818519 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-55d535fe-f046-41f2-b6bd-6c0ec1528f04 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:02:28.819935 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-55d535fe-f046-41f2-b6bd-6c0ec1528f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:28.879535 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1380/2773] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:02:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:02:28.887926 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c618ac91-f206-4037-ad7d-969396730398 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:28.888209 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c618ac91-f206-4037-ad7d-969396730398 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:28.888484 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c618ac91-f206-4037-ad7d-969396730398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:28.908583 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-c618ac91-f206-4037-ad7d-969396730398 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:02:28.910482 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-c618ac91-f206-4037-ad7d-969396730398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:28.918624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1394/2774] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:02:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:02:28.922847 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d535fe-f046-41f2-b6bd-6c0ec1528f04 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:28.932396 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d535fe-f046-41f2-b6bd-6c0ec1528f04 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:28.937037 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7729c07-fd5c-42dd-9b90-a2d80fbcca70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:28.937690 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7729c07-fd5c-42dd-9b90-a2d80fbcca70 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:28.937978 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7729c07-fd5c-42dd-9b90-a2d80fbcca70 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:28.938259 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7729c07-fd5c-42dd-9b90-a2d80fbcca70 admin admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:28.939018 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7729c07-fd5c-42dd-9b90-a2d80fbcca70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 23:02:28.939325 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7729c07-fd5c-42dd-9b90-a2d80fbcca70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:28.940906 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7729c07-fd5c-42dd-9b90-a2d80fbcca70 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:28.953566 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1381/2775] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 23:02:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 23:02:28.957642 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c618ac91-f206-4037-ad7d-969396730398 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:28.967698 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c618ac91-f206-4037-ad7d-969396730398 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:28.972359 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715b7146-dd6e-4e8e-842a-ef44f7ba1c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:28.973016 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715b7146-dd6e-4e8e-842a-ef44f7ba1c46 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:28.973302 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715b7146-dd6e-4e8e-842a-ef44f7ba1c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:28.973578 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715b7146-dd6e-4e8e-842a-ef44f7ba1c46 admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:28.974270 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715b7146-dd6e-4e8e-842a-ef44f7ba1c46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:28.976199 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715b7146-dd6e-4e8e-842a-ef44f7ba1c46 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:29.069547 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1395/2776] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 23:02:28 2022] POST /identity/v3/projects => generated 357 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:02:29.073625 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7729c07-fd5c-42dd-9b90-a2d80fbcca70 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:29.085233 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7729c07-fd5c-42dd-9b90-a2d80fbcca70 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:29.141533 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cfbc85-66fa-441d-8f2c-8598e9a9a8ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:29.142209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cfbc85-66fa-441d-8f2c-8598e9a9a8ba admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:29.142498 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cfbc85-66fa-441d-8f2c-8598e9a9a8ba admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:29.142794 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cfbc85-66fa-441d-8f2c-8598e9a9a8ba admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:29.143504 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cfbc85-66fa-441d-8f2c-8598e9a9a8ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:29.145440 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cfbc85-66fa-441d-8f2c-8598e9a9a8ba admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:29.217951 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1382/2777] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 23:02:29 2022] POST /identity/v3/users => generated 279 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 23:02:29.221008 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715b7146-dd6e-4e8e-842a-ef44f7ba1c46 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:29.232129 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715b7146-dd6e-4e8e-842a-ef44f7ba1c46 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:29.237317 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f486160d-5114-48e9-849d-5cf53d903758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:29.237910 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f486160d-5114-48e9-849d-5cf53d903758 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:29.238226 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f486160d-5114-48e9-849d-5cf53d903758 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:29.238501 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f486160d-5114-48e9-849d-5cf53d903758 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:29.239269 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f486160d-5114-48e9-849d-5cf53d903758 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:29.241588 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f486160d-5114-48e9-849d-5cf53d903758 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:29.252958 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1396/2778] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:02:29.257752 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cfbc85-66fa-441d-8f2c-8598e9a9a8ba admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:29.268258 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cfbc85-66fa-441d-8f2c-8598e9a9a8ba admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:29.278876 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0752e824-b234-468f-add0-639e86d81e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:29.279544 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752e824-b234-468f-add0-639e86d81e26 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:29.279824 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752e824-b234-468f-add0-639e86d81e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:29.280098 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752e824-b234-468f-add0-639e86d81e26 admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:29.280878 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0752e824-b234-468f-add0-639e86d81e26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:29.282500 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0752e824-b234-468f-add0-639e86d81e26 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:29.292807 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1383/2779] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 23:02:29.297170 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f486160d-5114-48e9-849d-5cf53d903758 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:29.307683 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f486160d-5114-48e9-849d-5cf53d903758 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:29.312242 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cef28f9-5959-415d-8ef3-b995980e30c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:29.313118 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef28f9-5959-415d-8ef3-b995980e30c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:29.313402 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef28f9-5959-415d-8ef3-b995980e30c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:29.313680 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef28f9-5959-415d-8ef3-b995980e30c5 admin admin] PATH_INFO: `/v3/projects/6d88e009d62f49e492947bf2d16e396c/users/62ec526058e24113b9f2b9b3bb5d1b54/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:29.328933 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cef28f9-5959-415d-8ef3-b995980e30c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d88e009d62f49e492947bf2d16e396c, user_id=62ec526058e24113b9f2b9b3bb5d1b54, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:29.331563 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cef28f9-5959-415d-8ef3-b995980e30c5 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:29.399567 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1397/2780] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 23:02:29 2022] PUT /identity/v3/projects/6d88e009d62f49e492947bf2d16e396c/users/62ec526058e24113b9f2b9b3bb5d1b54/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:29.411001 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9200e748-4560-4f46-b959-babce823573b None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:29.411312 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9200e748-4560-4f46-b959-babce823573b None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:29.411591 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9200e748-4560-4f46-b959-babce823573b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:29.438454 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-9200e748-4560-4f46-b959-babce823573b None None] MFA Rules not processed for user `62ec526058e24113b9f2b9b3bb5d1b54`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:02:29.439674 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-9200e748-4560-4f46-b959-babce823573b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:29.499897 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:29.505781 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1384/2781] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:02:29 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:02:29.510210 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cef28f9-5959-415d-8ef3-b995980e30c5 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:29.531477 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:29.573489 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cef28f9-5959-415d-8ef3-b995980e30c5 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:29.578621 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e181dc-53fa-481f-920d-70be787b4e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:29.579369 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e181dc-53fa-481f-920d-70be787b4e34 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:29.579720 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e181dc-53fa-481f-920d-70be787b4e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:29.580006 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e181dc-53fa-481f-920d-70be787b4e34 admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:29.580722 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e181dc-53fa-481f-920d-70be787b4e34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:29.582849 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e181dc-53fa-481f-920d-70be787b4e34 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:29.685426 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1398/2782] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 23:02:29 2022] POST /identity/v3/projects => generated 357 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:02:29.689342 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9200e748-4560-4f46-b959-babce823573b None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:29.700267 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9200e748-4560-4f46-b959-babce823573b None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:29.756858 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8e150-dd5d-4be0-8ab8-aaa8776e54fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:29.757533 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8e150-dd5d-4be0-8ab8-aaa8776e54fd admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:29.757817 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8e150-dd5d-4be0-8ab8-aaa8776e54fd admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:29.758094 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8e150-dd5d-4be0-8ab8-aaa8776e54fd admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:29.758793 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f8e150-dd5d-4be0-8ab8-aaa8776e54fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:29.760724 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f8e150-dd5d-4be0-8ab8-aaa8776e54fd admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:29.861024 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1385/2783] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 23:02:29 2022] POST /identity/v3/users => generated 285 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:02:29.865411 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e181dc-53fa-481f-920d-70be787b4e34 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:29.875964 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e181dc-53fa-481f-920d-70be787b4e34 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:29.880611 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52066e4-ff09-494e-9a57-9505d9d7a7a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:29.881270 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52066e4-ff09-494e-9a57-9505d9d7a7a1 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:29.881553 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52066e4-ff09-494e-9a57-9505d9d7a7a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:29.881831 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52066e4-ff09-494e-9a57-9505d9d7a7a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:29.882558 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52066e4-ff09-494e-9a57-9505d9d7a7a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:29.884198 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52066e4-ff09-494e-9a57-9505d9d7a7a1 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:29.894601 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1399/2784] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:02:29.898707 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8e150-dd5d-4be0-8ab8-aaa8776e54fd admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:29.908513 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8e150-dd5d-4be0-8ab8-aaa8776e54fd admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:29.913561 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf872c-8276-4769-89c0-456e98bc9211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:29.914435 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf872c-8276-4769-89c0-456e98bc9211 admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:29.914717 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf872c-8276-4769-89c0-456e98bc9211 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:29.915028 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf872c-8276-4769-89c0-456e98bc9211 admin admin] PATH_INFO: `/v3/projects/5a45c28db8ea44d3a6aa5700c1c09a31/users/645f4c996247467fb092a185530cac78/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:29.931376 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecf872c-8276-4769-89c0-456e98bc9211 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a45c28db8ea44d3a6aa5700c1c09a31, user_id=645f4c996247467fb092a185530cac78, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:29.933994 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecf872c-8276-4769-89c0-456e98bc9211 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:29.999427 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1386/2785] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 23:02:29 2022] PUT /identity/v3/projects/5a45c28db8ea44d3a6aa5700c1c09a31/users/645f4c996247467fb092a185530cac78/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:30.003361 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52066e4-ff09-494e-9a57-9505d9d7a7a1 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:30.013545 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52066e4-ff09-494e-9a57-9505d9d7a7a1 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:30.069317 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e5417-c8fd-44d0-a8d2-b2e5294c4f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:30.069976 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e5417-c8fd-44d0-a8d2-b2e5294c4f0a admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:30.070264 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e5417-c8fd-44d0-a8d2-b2e5294c4f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:30.070543 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e5417-c8fd-44d0-a8d2-b2e5294c4f0a admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:30.071296 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4e5417-c8fd-44d0-a8d2-b2e5294c4f0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:30.072883 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4e5417-c8fd-44d0-a8d2-b2e5294c4f0a admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:30.082445 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1400/2786] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:02:30 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:02:30.087022 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf872c-8276-4769-89c0-456e98bc9211 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:30.097137 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf872c-8276-4769-89c0-456e98bc9211 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:30.102439 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8874b0b6-ebe6-4404-9e08-3878992c0b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:30.103346 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874b0b6-ebe6-4404-9e08-3878992c0b5a admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:30.103638 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874b0b6-ebe6-4404-9e08-3878992c0b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:30.103916 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874b0b6-ebe6-4404-9e08-3878992c0b5a admin admin] PATH_INFO: `/v3/projects/5a45c28db8ea44d3a6aa5700c1c09a31/users/645f4c996247467fb092a185530cac78/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:30.106171 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8874b0b6-ebe6-4404-9e08-3878992c0b5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a45c28db8ea44d3a6aa5700c1c09a31, user_id=645f4c996247467fb092a185530cac78, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:30.108794 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8874b0b6-ebe6-4404-9e08-3878992c0b5a admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:30.119219 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1387/2787] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 23:02:30 2022] PUT /identity/v3/projects/5a45c28db8ea44d3a6aa5700c1c09a31/users/645f4c996247467fb092a185530cac78/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 23:02:30.129354 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4bb331-ceb6-48c8-b93c-54b35d2e4563 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:30.129635 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4bb331-ceb6-48c8-b93c-54b35d2e4563 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:30.129921 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4bb331-ceb6-48c8-b93c-54b35d2e4563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:30.158394 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-cc4bb331-ceb6-48c8-b93c-54b35d2e4563 None None] MFA Rules not processed for user `645f4c996247467fb092a185530cac78`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:02:30.159733 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-cc4bb331-ceb6-48c8-b93c-54b35d2e4563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:30.229592 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1401/2788] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:02:30 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:02:30.240661 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8874b0b6-ebe6-4404-9e08-3878992c0b5a admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:30.242375 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-8874b0b6-ebe6-4404-9e08-3878992c0b5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:30.301463 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8874b0b6-ebe6-4404-9e08-3878992c0b5a admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:30.305216 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dccccc5-c5e6-4547-b8dd-046556fea3f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:30.305919 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dccccc5-c5e6-4547-b8dd-046556fea3f9 service cinder] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:30.306222 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dccccc5-c5e6-4547-b8dd-046556fea3f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:30.306499 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dccccc5-c5e6-4547-b8dd-046556fea3f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:30.315691 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dccccc5-c5e6-4547-b8dd-046556fea3f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:30.317166 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dccccc5-c5e6-4547-b8dd-046556fea3f9 service cinder] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:30.326928 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1388/2789] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:02:30 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:02:30.571063 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4bb331-ceb6-48c8-b93c-54b35d2e4563 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:30.580703 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4bb331-ceb6-48c8-b93c-54b35d2e4563 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:30.584951 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1738937b-e77a-45d6-9c14-d342b346823d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:30.589951 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738937b-e77a-45d6-9c14-d342b346823d service cinder] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:30.589951 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738937b-e77a-45d6-9c14-d342b346823d service cinder] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:30.589951 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738937b-e77a-45d6-9c14-d342b346823d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:30.599020 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:30.647035 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1738937b-e77a-45d6-9c14-d342b346823d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:30.648571 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1738937b-e77a-45d6-9c14-d342b346823d service cinder] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:30.658016 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1402/2790] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:02:30 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:02:30.883027 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dccccc5-c5e6-4547-b8dd-046556fea3f9 service cinder] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:30.884469 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-7dccccc5-c5e6-4547-b8dd-046556fea3f9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:30.895660 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dccccc5-c5e6-4547-b8dd-046556fea3f9 service cinder] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:30.900318 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4a437a-82b9-4da9-b107-3f240f38b9a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:30.900318 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4a437a-82b9-4da9-b107-3f240f38b9a9 service cinder] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:30.900318 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4a437a-82b9-4da9-b107-3f240f38b9a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:30.900636 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4a437a-82b9-4da9-b107-3f240f38b9a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:30.902160 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-6b4a437a-82b9-4da9-b107-3f240f38b9a9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:30.951974 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:30.962554 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4a437a-82b9-4da9-b107-3f240f38b9a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:30.964024 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4a437a-82b9-4da9-b107-3f240f38b9a9 service cinder] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:30.975444 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1389/2791] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:02:30 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:02:32.420572 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1738937b-e77a-45d6-9c14-d342b346823d service cinder] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:32.429733 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1738937b-e77a-45d6-9c14-d342b346823d service cinder] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:32.433423 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29c2e00-4702-4164-9828-51f5d8bb5d81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:32.434065 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c2e00-4702-4164-9828-51f5d8bb5d81 service cinder] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:32.434349 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c2e00-4702-4164-9828-51f5d8bb5d81 service cinder] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:32.434624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c2e00-4702-4164-9828-51f5d8bb5d81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:32.436427 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-c29c2e00-4702-4164-9828-51f5d8bb5d81 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:32.509767 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29c2e00-4702-4164-9828-51f5d8bb5d81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:32.511042 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29c2e00-4702-4164-9828-51f5d8bb5d81 service cinder] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:32.522179 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1403/2792] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:02:32 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:02:34.260493 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4a437a-82b9-4da9-b107-3f240f38b9a9 service cinder] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:34.267737 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-6b4a437a-82b9-4da9-b107-3f240f38b9a9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:34.347167 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4a437a-82b9-4da9-b107-3f240f38b9a9 service cinder] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:34.355701 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03de5bf7-17a3-4d6b-9fd8-fbf5f4630de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:34.357048 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03de5bf7-17a3-4d6b-9fd8-fbf5f4630de1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:34.357864 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03de5bf7-17a3-4d6b-9fd8-fbf5f4630de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:34.358285 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03de5bf7-17a3-4d6b-9fd8-fbf5f4630de1 admin admin] PATH_INFO: `/v3/users/b7cca344a5b842bbb9ff9a3595c18bd4` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:34.364109 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03de5bf7-17a3-4d6b-9fd8-fbf5f4630de1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7cca344a5b842bbb9ff9a3595c18bd4)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:34.368810 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03de5bf7-17a3-4d6b-9fd8-fbf5f4630de1 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:34.606950 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-03de5bf7-17a3-4d6b-9fd8-fbf5f4630de1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7cca344a5b842bbb9ff9a3595c18bd4'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:34.614059 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-03de5bf7-17a3-4d6b-9fd8-fbf5f4630de1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7cca344a5b842bbb9ff9a3595c18bd4'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:34.614751 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-03de5bf7-17a3-4d6b-9fd8-fbf5f4630de1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7cca344a5b842bbb9ff9a3595c18bd4'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:34.628554 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-03de5bf7-17a3-4d6b-9fd8-fbf5f4630de1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7cca344a5b842bbb9ff9a3595c18bd4'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:34.629255 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-03de5bf7-17a3-4d6b-9fd8-fbf5f4630de1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7cca344a5b842bbb9ff9a3595c18bd4'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:34.730103 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1390/2793] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 23:02:34 2022] DELETE /identity/v3/users/b7cca344a5b842bbb9ff9a3595c18bd4 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:34.734590 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca18d67-cd72-4344-86c3-ce6e3a95e08a None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:34.735170 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca18d67-cd72-4344-86c3-ce6e3a95e08a None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:34.735271 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca18d67-cd72-4344-86c3-ce6e3a95e08a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:34.752766 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-8ca18d67-cd72-4344-86c3-ce6e3a95e08a None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:02:34.755976 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-8ca18d67-cd72-4344-86c3-ce6e3a95e08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:34.811360 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1404/2794] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:02:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:02:34.829734 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03de5bf7-17a3-4d6b-9fd8-fbf5f4630de1 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:34.829734 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-03de5bf7-17a3-4d6b-9fd8-fbf5f4630de1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:34.891553 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03de5bf7-17a3-4d6b-9fd8-fbf5f4630de1 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:34.895837 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58adff7-7309-4bc8-a828-bb727995c9ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:34.896479 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58adff7-7309-4bc8-a828-bb727995c9ab service neutron] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:34.897418 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58adff7-7309-4bc8-a828-bb727995c9ab service neutron] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:34.897712 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58adff7-7309-4bc8-a828-bb727995c9ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:34.907673 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58adff7-7309-4bc8-a828-bb727995c9ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:34.909093 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58adff7-7309-4bc8-a828-bb727995c9ab service neutron] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:34.918451 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1391/2795] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:02:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:02:35.338594 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca18d67-cd72-4344-86c3-ce6e3a95e08a None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:35.340159 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-8ca18d67-cd72-4344-86c3-ce6e3a95e08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:35.352305 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca18d67-cd72-4344-86c3-ce6e3a95e08a None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:35.357402 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d037793e-748b-49bb-b4fe-a71a980d728e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:35.358250 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d037793e-748b-49bb-b4fe-a71a980d728e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:35.358528 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d037793e-748b-49bb-b4fe-a71a980d728e admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:35.358858 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d037793e-748b-49bb-b4fe-a71a980d728e admin admin] PATH_INFO: `/v3/projects/2b332ea671ec4c13b56113a2e7d30e31` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:35.360429 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d037793e-748b-49bb-b4fe-a71a980d728e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b332ea671ec4c13b56113a2e7d30e31)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:35.362429 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d037793e-748b-49bb-b4fe-a71a980d728e admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:35.485945 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:35.490272 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-d037793e-748b-49bb-b4fe-a71a980d728e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b332ea671ec4c13b56113a2e7d30e31', 'request_id': 'req-d037793e-748b-49bb-b4fe-a71a980d728e'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:35.504675 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-d037793e-748b-49bb-b4fe-a71a980d728e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b332ea671ec4c13b56113a2e7d30e31', 'request_id': 'req-d037793e-748b-49bb-b4fe-a71a980d728e'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:35.572325 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-d037793e-748b-49bb-b4fe-a71a980d728e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b332ea671ec4c13b56113a2e7d30e31', 'request_id': 'req-d037793e-748b-49bb-b4fe-a71a980d728e'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:35.586430 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-d037793e-748b-49bb-b4fe-a71a980d728e admin admin] The token cache is being invalidate because project 2b332ea671ec4c13b56113a2e7d30e31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:02:35.586725 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-d037793e-748b-49bb-b4fe-a71a980d728e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:35.587489 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-d037793e-748b-49bb-b4fe-a71a980d728e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:35.589257 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1405/2796] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 23:02:35 2022] DELETE /identity/v3/projects/2b332ea671ec4c13b56113a2e7d30e31 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:37.218485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58adff7-7309-4bc8-a828-bb727995c9ab service neutron] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:37.219956 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-b58adff7-7309-4bc8-a828-bb727995c9ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:37.233200 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58adff7-7309-4bc8-a828-bb727995c9ab service neutron] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:37.242049 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:37.293305 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f909837a-d1da-4a2c-98b7-a75f84a4f91c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:37.294173 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909837a-d1da-4a2c-98b7-a75f84a4f91c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:37.294456 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909837a-d1da-4a2c-98b7-a75f84a4f91c admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:37.294815 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909837a-d1da-4a2c-98b7-a75f84a4f91c admin admin] PATH_INFO: `/v3/users/ccd0d42a04324e58a3cbca7c5729bdc5` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:37.296026 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f909837a-d1da-4a2c-98b7-a75f84a4f91c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccd0d42a04324e58a3cbca7c5729bdc5)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:37.298033 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f909837a-d1da-4a2c-98b7-a75f84a4f91c admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:37.511681 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-f909837a-d1da-4a2c-98b7-a75f84a4f91c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccd0d42a04324e58a3cbca7c5729bdc5'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:37.518759 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-f909837a-d1da-4a2c-98b7-a75f84a4f91c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccd0d42a04324e58a3cbca7c5729bdc5'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:37.519475 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-f909837a-d1da-4a2c-98b7-a75f84a4f91c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccd0d42a04324e58a3cbca7c5729bdc5'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:37.531513 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-f909837a-d1da-4a2c-98b7-a75f84a4f91c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccd0d42a04324e58a3cbca7c5729bdc5'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:37.534712 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-f909837a-d1da-4a2c-98b7-a75f84a4f91c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccd0d42a04324e58a3cbca7c5729bdc5'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:37.616865 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1392/2797] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 23:02:37 2022] DELETE /identity/v3/users/ccd0d42a04324e58a3cbca7c5729bdc5 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:37.621935 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3ac2fd-56c7-48d9-afc2-240d24930fa5 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:37.622400 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3ac2fd-56c7-48d9-afc2-240d24930fa5 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:37.622521 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3ac2fd-56c7-48d9-afc2-240d24930fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:37.640962 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-bf3ac2fd-56c7-48d9-afc2-240d24930fa5 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:02:37.642168 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-bf3ac2fd-56c7-48d9-afc2-240d24930fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:37.701857 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1406/2798] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:02:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:02:37.716150 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f909837a-d1da-4a2c-98b7-a75f84a4f91c admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:37.717563 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-f909837a-d1da-4a2c-98b7-a75f84a4f91c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:37.775395 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f909837a-d1da-4a2c-98b7-a75f84a4f91c admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:37.778969 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de4739d-9134-466a-a642-d4ea34a99837 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:37.779612 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de4739d-9134-466a-a642-d4ea34a99837 service neutron] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:37.779893 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de4739d-9134-466a-a642-d4ea34a99837 service neutron] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:37.780168 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de4739d-9134-466a-a642-d4ea34a99837 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:37.789508 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de4739d-9134-466a-a642-d4ea34a99837 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:37.790945 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de4739d-9134-466a-a642-d4ea34a99837 service neutron] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:37.800341 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1393/2799] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:02:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:02:38.207238 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3ac2fd-56c7-48d9-afc2-240d24930fa5 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:38.208675 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-bf3ac2fd-56c7-48d9-afc2-240d24930fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:38.222426 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3ac2fd-56c7-48d9-afc2-240d24930fa5 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:38.227937 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61cce00-9333-4f0b-807d-5369a5e83e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:38.233018 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61cce00-9333-4f0b-807d-5369a5e83e62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:38.233018 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61cce00-9333-4f0b-807d-5369a5e83e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:38.233018 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61cce00-9333-4f0b-807d-5369a5e83e62 admin admin] PATH_INFO: `/v3/projects/3dd7349d012948619faf1671b6637f1a` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:38.233018 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61cce00-9333-4f0b-807d-5369a5e83e62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dd7349d012948619faf1671b6637f1a)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:38.233852 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61cce00-9333-4f0b-807d-5369a5e83e62 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:38.344744 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-a61cce00-9333-4f0b-807d-5369a5e83e62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dd7349d012948619faf1671b6637f1a', 'request_id': 'req-a61cce00-9333-4f0b-807d-5369a5e83e62'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:38.350803 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-a61cce00-9333-4f0b-807d-5369a5e83e62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dd7349d012948619faf1671b6637f1a', 'request_id': 'req-a61cce00-9333-4f0b-807d-5369a5e83e62'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:38.428328 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-a61cce00-9333-4f0b-807d-5369a5e83e62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dd7349d012948619faf1671b6637f1a', 'request_id': 'req-a61cce00-9333-4f0b-807d-5369a5e83e62'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:38.448423 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-a61cce00-9333-4f0b-807d-5369a5e83e62 admin admin] The token cache is being invalidate because project 3dd7349d012948619faf1671b6637f1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:02:38.448933 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-a61cce00-9333-4f0b-807d-5369a5e83e62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:38.450057 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-a61cce00-9333-4f0b-807d-5369a5e83e62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:38.452267 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1407/2800] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 23:02:38 2022] DELETE /identity/v3/projects/3dd7349d012948619faf1671b6637f1a => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:38.456049 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de4739d-9134-466a-a642-d4ea34a99837 service neutron] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:38.466821 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de4739d-9134-466a-a642-d4ea34a99837 service neutron] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:38.506346 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:38.534793 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7603f2d-ee1a-4e32-9614-b2fc7ab4316d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:38.535925 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7603f2d-ee1a-4e32-9614-b2fc7ab4316d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:38.536434 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7603f2d-ee1a-4e32-9614-b2fc7ab4316d admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:38.536898 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7603f2d-ee1a-4e32-9614-b2fc7ab4316d admin admin] PATH_INFO: `/v3/users/7a9e9c34faef44d3a9e7303d7494f2d7` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:38.538678 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7603f2d-ee1a-4e32-9614-b2fc7ab4316d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a9e9c34faef44d3a9e7303d7494f2d7)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:38.541293 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7603f2d-ee1a-4e32-9614-b2fc7ab4316d admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:38.736864 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-d7603f2d-ee1a-4e32-9614-b2fc7ab4316d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a9e9c34faef44d3a9e7303d7494f2d7'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:38.745461 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-d7603f2d-ee1a-4e32-9614-b2fc7ab4316d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a9e9c34faef44d3a9e7303d7494f2d7'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:38.746241 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-d7603f2d-ee1a-4e32-9614-b2fc7ab4316d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a9e9c34faef44d3a9e7303d7494f2d7'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:38.770537 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-d7603f2d-ee1a-4e32-9614-b2fc7ab4316d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a9e9c34faef44d3a9e7303d7494f2d7'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:38.771290 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-d7603f2d-ee1a-4e32-9614-b2fc7ab4316d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a9e9c34faef44d3a9e7303d7494f2d7'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:38.883942 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1394/2801] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 23:02:38 2022] DELETE /identity/v3/users/7a9e9c34faef44d3a9e7303d7494f2d7 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:39.373349 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61cce00-9333-4f0b-807d-5369a5e83e62 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:39.374899 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-a61cce00-9333-4f0b-807d-5369a5e83e62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:39.388348 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61cce00-9333-4f0b-807d-5369a5e83e62 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:39.440339 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:39.447168 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba40dea-3b03-42d7-b583-413642330e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:39.448032 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba40dea-3b03-42d7-b583-413642330e93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:39.448321 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba40dea-3b03-42d7-b583-413642330e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:39.448601 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba40dea-3b03-42d7-b583-413642330e93 admin admin] PATH_INFO: `/v3/projects/4021f07357d24ea99e72dce9e94281b2` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:39.449859 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba40dea-3b03-42d7-b583-413642330e93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4021f07357d24ea99e72dce9e94281b2)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:39.451791 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba40dea-3b03-42d7-b583-413642330e93 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:39.604775 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-0ba40dea-3b03-42d7-b583-413642330e93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4021f07357d24ea99e72dce9e94281b2', 'request_id': 'req-0ba40dea-3b03-42d7-b583-413642330e93'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:39.610204 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-0ba40dea-3b03-42d7-b583-413642330e93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4021f07357d24ea99e72dce9e94281b2', 'request_id': 'req-0ba40dea-3b03-42d7-b583-413642330e93'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:39.662140 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-0ba40dea-3b03-42d7-b583-413642330e93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4021f07357d24ea99e72dce9e94281b2', 'request_id': 'req-0ba40dea-3b03-42d7-b583-413642330e93'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:39.674228 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-0ba40dea-3b03-42d7-b583-413642330e93 admin admin] The token cache is being invalidate because project 4021f07357d24ea99e72dce9e94281b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:02:39.674516 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-0ba40dea-3b03-42d7-b583-413642330e93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:39.675165 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-0ba40dea-3b03-42d7-b583-413642330e93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:39.676697 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1408/2802] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 23:02:39 2022] DELETE /identity/v3/projects/4021f07357d24ea99e72dce9e94281b2 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:39.680666 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7603f2d-ee1a-4e32-9614-b2fc7ab4316d admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:39.682120 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-d7603f2d-ee1a-4e32-9614-b2fc7ab4316d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:39.695018 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7603f2d-ee1a-4e32-9614-b2fc7ab4316d admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:39.760230 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17050cc-6cdf-4d5a-ba2f-1653f543b7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:39.761146 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17050cc-6cdf-4d5a-ba2f-1653f543b7d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:39.761521 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17050cc-6cdf-4d5a-ba2f-1653f543b7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:39.761805 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17050cc-6cdf-4d5a-ba2f-1653f543b7d7 admin admin] PATH_INFO: `/v3/users/ab57ec5020a74ba0af0dc4c03ff89568` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:39.763168 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17050cc-6cdf-4d5a-ba2f-1653f543b7d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab57ec5020a74ba0af0dc4c03ff89568)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:39.765283 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17050cc-6cdf-4d5a-ba2f-1653f543b7d7 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:39.989450 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-c17050cc-6cdf-4d5a-ba2f-1653f543b7d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab57ec5020a74ba0af0dc4c03ff89568'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:39.996964 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-c17050cc-6cdf-4d5a-ba2f-1653f543b7d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab57ec5020a74ba0af0dc4c03ff89568'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:39.997623 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-c17050cc-6cdf-4d5a-ba2f-1653f543b7d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab57ec5020a74ba0af0dc4c03ff89568'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:39.998752 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:40.010454 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-c17050cc-6cdf-4d5a-ba2f-1653f543b7d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab57ec5020a74ba0af0dc4c03ff89568'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:40.011083 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-c17050cc-6cdf-4d5a-ba2f-1653f543b7d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab57ec5020a74ba0af0dc4c03ff89568'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:40.060657 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1395/2803] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 23:02:39 2022] DELETE /identity/v3/users/ab57ec5020a74ba0af0dc4c03ff89568 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:40.556344 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba40dea-3b03-42d7-b583-413642330e93 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:40.557909 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-0ba40dea-3b03-42d7-b583-413642330e93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:40.573344 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba40dea-3b03-42d7-b583-413642330e93 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:40.632319 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e78d71-7e98-4c6f-833f-5ac863ebbc7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:40.633764 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e78d71-7e98-4c6f-833f-5ac863ebbc7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:40.634046 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e78d71-7e98-4c6f-833f-5ac863ebbc7c admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:40.634323 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e78d71-7e98-4c6f-833f-5ac863ebbc7c admin admin] PATH_INFO: `/v3/projects/eaee93ac6ae3461da819e4cf729cf9b7` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:40.636073 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e78d71-7e98-4c6f-833f-5ac863ebbc7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaee93ac6ae3461da819e4cf729cf9b7)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:40.637960 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e78d71-7e98-4c6f-833f-5ac863ebbc7c admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:40.788389 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-e1e78d71-7e98-4c6f-833f-5ac863ebbc7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaee93ac6ae3461da819e4cf729cf9b7', 'request_id': 'req-e1e78d71-7e98-4c6f-833f-5ac863ebbc7c'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:40.794292 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-e1e78d71-7e98-4c6f-833f-5ac863ebbc7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaee93ac6ae3461da819e4cf729cf9b7', 'request_id': 'req-e1e78d71-7e98-4c6f-833f-5ac863ebbc7c'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:40.857719 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-e1e78d71-7e98-4c6f-833f-5ac863ebbc7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaee93ac6ae3461da819e4cf729cf9b7', 'request_id': 'req-e1e78d71-7e98-4c6f-833f-5ac863ebbc7c'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:40.873694 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-e1e78d71-7e98-4c6f-833f-5ac863ebbc7c admin admin] The token cache is being invalidate because project eaee93ac6ae3461da819e4cf729cf9b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:02:40.873992 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-e1e78d71-7e98-4c6f-833f-5ac863ebbc7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:40.878366 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-e1e78d71-7e98-4c6f-833f-5ac863ebbc7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:40.890686 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1409/2804] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 23:02:40 2022] DELETE /identity/v3/projects/eaee93ac6ae3461da819e4cf729cf9b7 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:49.477949 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17050cc-6cdf-4d5a-ba2f-1653f543b7d7 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:49.477949 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-c17050cc-6cdf-4d5a-ba2f-1653f543b7d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:49.490405 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17050cc-6cdf-4d5a-ba2f-1653f543b7d7 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:49.548469 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04c596c-300d-4897-89ad-f37084e18bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:49.549356 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04c596c-300d-4897-89ad-f37084e18bee admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:49.549647 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04c596c-300d-4897-89ad-f37084e18bee admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:49.549925 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04c596c-300d-4897-89ad-f37084e18bee admin admin] PATH_INFO: `/v3/users/62ec526058e24113b9f2b9b3bb5d1b54` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:49.551196 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04c596c-300d-4897-89ad-f37084e18bee admin admin] RBAC: Authorizing `identity:delete_user(user_id=62ec526058e24113b9f2b9b3bb5d1b54)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:49.553138 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04c596c-300d-4897-89ad-f37084e18bee admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:49.736851 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-c04c596c-300d-4897-89ad-f37084e18bee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62ec526058e24113b9f2b9b3bb5d1b54'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:49.743999 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-c04c596c-300d-4897-89ad-f37084e18bee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62ec526058e24113b9f2b9b3bb5d1b54'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:49.744590 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-c04c596c-300d-4897-89ad-f37084e18bee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62ec526058e24113b9f2b9b3bb5d1b54'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:49.756787 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-c04c596c-300d-4897-89ad-f37084e18bee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62ec526058e24113b9f2b9b3bb5d1b54'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:49.757400 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-c04c596c-300d-4897-89ad-f37084e18bee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62ec526058e24113b9f2b9b3bb5d1b54'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:49.823391 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1396/2805] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 23:02:49 2022] DELETE /identity/v3/users/62ec526058e24113b9f2b9b3bb5d1b54 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:49.828038 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356a3884-3302-484f-9b4c-75044e03a831 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:49.828486 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356a3884-3302-484f-9b4c-75044e03a831 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:49.828585 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356a3884-3302-484f-9b4c-75044e03a831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:49.847663 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-356a3884-3302-484f-9b4c-75044e03a831 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:02:49.848867 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-356a3884-3302-484f-9b4c-75044e03a831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:49.859757 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:49.910892 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1410/2806] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:02:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:02:49.922164 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04c596c-300d-4897-89ad-f37084e18bee admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:49.923943 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-c04c596c-300d-4897-89ad-f37084e18bee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:49.987707 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04c596c-300d-4897-89ad-f37084e18bee admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:49.991634 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29077fb1-163a-498b-9790-d9faff986e98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:49.992294 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29077fb1-163a-498b-9790-d9faff986e98 service neutron] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:49.992583 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29077fb1-163a-498b-9790-d9faff986e98 service neutron] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:49.992865 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29077fb1-163a-498b-9790-d9faff986e98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:50.000314 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:50.002539 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29077fb1-163a-498b-9790-d9faff986e98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:50.004008 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29077fb1-163a-498b-9790-d9faff986e98 service neutron] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:50.014017 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1397/2807] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:02:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:02:50.436548 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356a3884-3302-484f-9b4c-75044e03a831 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:50.438128 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-356a3884-3302-484f-9b4c-75044e03a831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:50.451468 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356a3884-3302-484f-9b4c-75044e03a831 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:50.456142 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1559712e-8747-4661-8984-887db5c9f323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:50.457129 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1559712e-8747-4661-8984-887db5c9f323 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:50.457508 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1559712e-8747-4661-8984-887db5c9f323 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:50.457792 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1559712e-8747-4661-8984-887db5c9f323 admin admin] PATH_INFO: `/v3/projects/6d88e009d62f49e492947bf2d16e396c` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:50.459296 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1559712e-8747-4661-8984-887db5c9f323 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d88e009d62f49e492947bf2d16e396c)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:50.461688 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1559712e-8747-4661-8984-887db5c9f323 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:50.597181 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-1559712e-8747-4661-8984-887db5c9f323 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d88e009d62f49e492947bf2d16e396c', 'request_id': 'req-1559712e-8747-4661-8984-887db5c9f323'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:50.603652 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-1559712e-8747-4661-8984-887db5c9f323 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d88e009d62f49e492947bf2d16e396c', 'request_id': 'req-1559712e-8747-4661-8984-887db5c9f323'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:50.662835 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-1559712e-8747-4661-8984-887db5c9f323 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d88e009d62f49e492947bf2d16e396c', 'request_id': 'req-1559712e-8747-4661-8984-887db5c9f323'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:50.675324 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-1559712e-8747-4661-8984-887db5c9f323 admin admin] The token cache is being invalidate because project 6d88e009d62f49e492947bf2d16e396c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:02:50.675611 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-1559712e-8747-4661-8984-887db5c9f323 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:50.676236 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-1559712e-8747-4661-8984-887db5c9f323 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:50.677885 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1411/2808] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 23:02:50 2022] DELETE /identity/v3/projects/6d88e009d62f49e492947bf2d16e396c => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:50.684001 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29077fb1-163a-498b-9790-d9faff986e98 service neutron] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:50.693738 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29077fb1-163a-498b-9790-d9faff986e98 service neutron] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:50.751872 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d3b379-1420-4b95-b74a-7fd7c08f25d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:50.752769 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d3b379-1420-4b95-b74a-7fd7c08f25d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:50.753052 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d3b379-1420-4b95-b74a-7fd7c08f25d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:50.753329 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d3b379-1420-4b95-b74a-7fd7c08f25d2 admin admin] PATH_INFO: `/v3/users/645f4c996247467fb092a185530cac78` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:50.754513 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d3b379-1420-4b95-b74a-7fd7c08f25d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=645f4c996247467fb092a185530cac78)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:50.757169 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d3b379-1420-4b95-b74a-7fd7c08f25d2 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:50.931772 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-65d3b379-1420-4b95-b74a-7fd7c08f25d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '645f4c996247467fb092a185530cac78'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:50.938644 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-65d3b379-1420-4b95-b74a-7fd7c08f25d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '645f4c996247467fb092a185530cac78'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:50.939335 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-65d3b379-1420-4b95-b74a-7fd7c08f25d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '645f4c996247467fb092a185530cac78'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:50.951363 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-65d3b379-1420-4b95-b74a-7fd7c08f25d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '645f4c996247467fb092a185530cac78'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:50.951993 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-65d3b379-1420-4b95-b74a-7fd7c08f25d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '645f4c996247467fb092a185530cac78'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:51.020540 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1398/2809] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 23:02:50 2022] DELETE /identity/v3/users/645f4c996247467fb092a185530cac78 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:51.430076 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1559712e-8747-4661-8984-887db5c9f323 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:51.431573 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-1559712e-8747-4661-8984-887db5c9f323 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:51.445520 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1559712e-8747-4661-8984-887db5c9f323 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:51.510374 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472390a7-deb9-4676-8bd3-ceb70ce8fbc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:51.511292 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472390a7-deb9-4676-8bd3-ceb70ce8fbc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:51.511576 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472390a7-deb9-4676-8bd3-ceb70ce8fbc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:51.511850 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472390a7-deb9-4676-8bd3-ceb70ce8fbc1 admin admin] PATH_INFO: `/v3/projects/5a45c28db8ea44d3a6aa5700c1c09a31` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:51.513134 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472390a7-deb9-4676-8bd3-ceb70ce8fbc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a45c28db8ea44d3a6aa5700c1c09a31)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:51.515101 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472390a7-deb9-4676-8bd3-ceb70ce8fbc1 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:51.635326 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:51.642676 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-472390a7-deb9-4676-8bd3-ceb70ce8fbc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a45c28db8ea44d3a6aa5700c1c09a31', 'request_id': 'req-472390a7-deb9-4676-8bd3-ceb70ce8fbc1'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:51.648745 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-472390a7-deb9-4676-8bd3-ceb70ce8fbc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a45c28db8ea44d3a6aa5700c1c09a31', 'request_id': 'req-472390a7-deb9-4676-8bd3-ceb70ce8fbc1'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:51.725862 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-472390a7-deb9-4676-8bd3-ceb70ce8fbc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a45c28db8ea44d3a6aa5700c1c09a31', 'request_id': 'req-472390a7-deb9-4676-8bd3-ceb70ce8fbc1'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:51.739616 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-472390a7-deb9-4676-8bd3-ceb70ce8fbc1 admin admin] The token cache is being invalidate because project 5a45c28db8ea44d3a6aa5700c1c09a31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:02:51.739903 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-472390a7-deb9-4676-8bd3-ceb70ce8fbc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:51.740634 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-472390a7-deb9-4676-8bd3-ceb70ce8fbc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:51.742173 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1412/2810] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 23:02:51 2022] DELETE /identity/v3/projects/5a45c28db8ea44d3a6aa5700c1c09a31 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:51.747743 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf5db2-4169-45ed-9182-e1c24f6a294d None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:51.748200 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf5db2-4169-45ed-9182-e1c24f6a294d None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:51.748315 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf5db2-4169-45ed-9182-e1c24f6a294d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:51.766493 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-50cf5db2-4169-45ed-9182-e1c24f6a294d None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:02:51.767783 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-50cf5db2-4169-45ed-9182-e1c24f6a294d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:51.829556 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1399/2811] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:02:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:02:51.891829 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5a964-d596-47b0-9098-bfbefa9fd384 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:51.892119 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5a964-d596-47b0-9098-bfbefa9fd384 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:51.892399 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5a964-d596-47b0-9098-bfbefa9fd384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:51.911401 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-16f5a964-d596-47b0-9098-bfbefa9fd384 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:02:51.912661 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-16f5a964-d596-47b0-9098-bfbefa9fd384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:51.920891 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1413/2812] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:02:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:02:51.925246 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cf5db2-4169-45ed-9182-e1c24f6a294d None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:51.937015 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cf5db2-4169-45ed-9182-e1c24f6a294d None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:51.942405 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379decf9-b18d-4543-a9ce-9a923299fa21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:51.943098 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379decf9-b18d-4543-a9ce-9a923299fa21 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:51.943581 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379decf9-b18d-4543-a9ce-9a923299fa21 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:51.944089 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379decf9-b18d-4543-a9ce-9a923299fa21 admin admin] PATH_INFO: `/v3/domains` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:51.945442 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379decf9-b18d-4543-a9ce-9a923299fa21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 23:02:51.945887 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379decf9-b18d-4543-a9ce-9a923299fa21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:51.948101 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379decf9-b18d-4543-a9ce-9a923299fa21 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:51.967377 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1400/2813] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 23:02:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 23:02:51.971781 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f5a964-d596-47b0-9098-bfbefa9fd384 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:51.981763 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f5a964-d596-47b0-9098-bfbefa9fd384 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:51.986498 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee74f5d-0ab3-4860-b936-2be88b1ca423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:51.987176 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee74f5d-0ab3-4860-b936-2be88b1ca423 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:51.987465 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee74f5d-0ab3-4860-b936-2be88b1ca423 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:51.987743 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee74f5d-0ab3-4860-b936-2be88b1ca423 admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:51.988444 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee74f5d-0ab3-4860-b936-2be88b1ca423 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:51.990473 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee74f5d-0ab3-4860-b936-2be88b1ca423 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:52.127691 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1414/2814] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 23:02:51 2022] POST /identity/v3/projects => generated 387 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:02:52.131952 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379decf9-b18d-4543-a9ce-9a923299fa21 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:52.137235 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:52.142766 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379decf9-b18d-4543-a9ce-9a923299fa21 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:52.200490 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec703e-5828-42ae-9e0b-cd03adc89ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:52.201167 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec703e-5828-42ae-9e0b-cd03adc89ee8 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:52.201452 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec703e-5828-42ae-9e0b-cd03adc89ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:52.201732 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec703e-5828-42ae-9e0b-cd03adc89ee8 admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:52.202424 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dec703e-5828-42ae-9e0b-cd03adc89ee8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:52.204558 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dec703e-5828-42ae-9e0b-cd03adc89ee8 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:52.772495 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1401/2815] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 23:02:52 2022] POST /identity/v3/users => generated 294 bytes in 641 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:02:52.777176 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee74f5d-0ab3-4860-b936-2be88b1ca423 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:52.788445 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee74f5d-0ab3-4860-b936-2be88b1ca423 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:52.793579 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510d2070-7607-42ec-885a-6f6ddbc4a659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:52.797028 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510d2070-7607-42ec-885a-6f6ddbc4a659 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:52.797028 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510d2070-7607-42ec-885a-6f6ddbc4a659 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:52.797028 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510d2070-7607-42ec-885a-6f6ddbc4a659 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:52.797028 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510d2070-7607-42ec-885a-6f6ddbc4a659 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:52.797569 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510d2070-7607-42ec-885a-6f6ddbc4a659 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:52.808252 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1415/2816] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:02:52.813072 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec703e-5828-42ae-9e0b-cd03adc89ee8 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:52.823134 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec703e-5828-42ae-9e0b-cd03adc89ee8 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:52.827633 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa92958f-7db9-41e4-9e1c-e37c21bcfd62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:52.828307 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa92958f-7db9-41e4-9e1c-e37c21bcfd62 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:52.828580 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa92958f-7db9-41e4-9e1c-e37c21bcfd62 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:52.828861 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa92958f-7db9-41e4-9e1c-e37c21bcfd62 admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:52.829589 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa92958f-7db9-41e4-9e1c-e37c21bcfd62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:52.831213 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa92958f-7db9-41e4-9e1c-e37c21bcfd62 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:52.841138 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1402/2817] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:02:52.845474 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510d2070-7607-42ec-885a-6f6ddbc4a659 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:52.857012 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510d2070-7607-42ec-885a-6f6ddbc4a659 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:52.861850 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ce0ccd-2bb1-42d1-bfde-0717de31847c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:52.862754 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce0ccd-2bb1-42d1-bfde-0717de31847c admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:52.863104 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce0ccd-2bb1-42d1-bfde-0717de31847c admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:52.863384 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce0ccd-2bb1-42d1-bfde-0717de31847c admin admin] PATH_INFO: `/v3/projects/ddf3224e839c48cb8a760afd3cb2c3b2/users/b8533e19fbc5480291472bb103e656f3/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:52.878999 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ce0ccd-2bb1-42d1-bfde-0717de31847c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddf3224e839c48cb8a760afd3cb2c3b2, user_id=b8533e19fbc5480291472bb103e656f3, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:52.883742 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ce0ccd-2bb1-42d1-bfde-0717de31847c admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:52.948147 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1416/2818] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 23:02:52 2022] PUT /identity/v3/projects/ddf3224e839c48cb8a760afd3cb2c3b2/users/b8533e19fbc5480291472bb103e656f3/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:52.959662 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9144e970-4b11-45e9-b5d7-e40a3b0ef6ea None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:52.959951 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9144e970-4b11-45e9-b5d7-e40a3b0ef6ea None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:52.960228 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9144e970-4b11-45e9-b5d7-e40a3b0ef6ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:52.987603 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-9144e970-4b11-45e9-b5d7-e40a3b0ef6ea None None] MFA Rules not processed for user `b8533e19fbc5480291472bb103e656f3`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:02:52.988813 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-9144e970-4b11-45e9-b5d7-e40a3b0ef6ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:53.056993 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1403/2819] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:02:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:02:53.061463 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ce0ccd-2bb1-42d1-bfde-0717de31847c admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:53.124883 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ce0ccd-2bb1-42d1-bfde-0717de31847c admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:53.129560 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c76d39-92dd-4c97-b6a8-331f28de7267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:53.130220 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c76d39-92dd-4c97-b6a8-331f28de7267 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:53.130509 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c76d39-92dd-4c97-b6a8-331f28de7267 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:53.130880 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c76d39-92dd-4c97-b6a8-331f28de7267 admin admin] PATH_INFO: `/v3/projects` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:53.131590 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c76d39-92dd-4c97-b6a8-331f28de7267 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:53.133517 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c76d39-92dd-4c97-b6a8-331f28de7267 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:53.233358 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1417/2820] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 23:02:53 2022] POST /identity/v3/projects => generated 385 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:02:53.235406 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9144e970-4b11-45e9-b5d7-e40a3b0ef6ea None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:53.246551 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9144e970-4b11-45e9-b5d7-e40a3b0ef6ea None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:53.304889 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95853ec-28bb-49d3-82b5-9668e64475f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:53.305624 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95853ec-28bb-49d3-82b5-9668e64475f6 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:53.305910 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95853ec-28bb-49d3-82b5-9668e64475f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:53.306191 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95853ec-28bb-49d3-82b5-9668e64475f6 admin admin] PATH_INFO: `/v3/users` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:53.306941 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95853ec-28bb-49d3-82b5-9668e64475f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:53.309010 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95853ec-28bb-49d3-82b5-9668e64475f6 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:53.384311 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:53.390370 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1404/2821] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 23:02:53 2022] POST /identity/v3/users => generated 299 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 23:02:53.394345 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c76d39-92dd-4c97-b6a8-331f28de7267 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:53.405388 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c76d39-92dd-4c97-b6a8-331f28de7267 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:53.410613 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09972ca-b9ec-4b51-b6fb-1c727d5bedf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:53.411305 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09972ca-b9ec-4b51-b6fb-1c727d5bedf3 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:53.411588 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09972ca-b9ec-4b51-b6fb-1c727d5bedf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:53.411903 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09972ca-b9ec-4b51-b6fb-1c727d5bedf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:53.412628 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09972ca-b9ec-4b51-b6fb-1c727d5bedf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:53.414214 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09972ca-b9ec-4b51-b6fb-1c727d5bedf3 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:53.427585 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1418/2822] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:02:53.431733 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95853ec-28bb-49d3-82b5-9668e64475f6 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:53.442552 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95853ec-28bb-49d3-82b5-9668e64475f6 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:53.447982 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba913f09-a082-4ace-b99a-2e61209806b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:53.451613 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba913f09-a082-4ace-b99a-2e61209806b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:53.451613 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba913f09-a082-4ace-b99a-2e61209806b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:53.451613 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba913f09-a082-4ace-b99a-2e61209806b4 admin admin] PATH_INFO: `/v3/projects/fe9673034438436880d63ac5d3222b42/users/b49dd21d74454a56a0e84562160ef20b/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:53.465931 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba913f09-a082-4ace-b99a-2e61209806b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe9673034438436880d63ac5d3222b42, user_id=b49dd21d74454a56a0e84562160ef20b, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:53.468821 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba913f09-a082-4ace-b99a-2e61209806b4 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:53.593192 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1405/2823] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 23:02:53 2022] PUT /identity/v3/projects/fe9673034438436880d63ac5d3222b42/users/b49dd21d74454a56a0e84562160ef20b/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:53.597250 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09972ca-b9ec-4b51-b6fb-1c727d5bedf3 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:53.608085 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09972ca-b9ec-4b51-b6fb-1c727d5bedf3 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:53.649234 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:53.667921 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85c900a-a4dd-4e7a-995d-5069a15722a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:53.668740 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c900a-a4dd-4e7a-995d-5069a15722a4 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:53.669026 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c900a-a4dd-4e7a-995d-5069a15722a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:53.669325 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c900a-a4dd-4e7a-995d-5069a15722a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:53.670066 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85c900a-a4dd-4e7a-995d-5069a15722a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:53.671915 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85c900a-a4dd-4e7a-995d-5069a15722a4 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:53.683003 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1419/2824] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:02:53.686987 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba913f09-a082-4ace-b99a-2e61209806b4 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:53.697491 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba913f09-a082-4ace-b99a-2e61209806b4 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:53.702105 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e889f-8f60-4e5d-b055-1781346c7462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:53.703006 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e889f-8f60-4e5d-b055-1781346c7462 admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:53.703289 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e889f-8f60-4e5d-b055-1781346c7462 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:53.703561 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e889f-8f60-4e5d-b055-1781346c7462 admin admin] PATH_INFO: `/v3/projects/fe9673034438436880d63ac5d3222b42/users/b49dd21d74454a56a0e84562160ef20b/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:53.705497 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36e889f-8f60-4e5d-b055-1781346c7462 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe9673034438436880d63ac5d3222b42, user_id=b49dd21d74454a56a0e84562160ef20b, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:53.708329 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36e889f-8f60-4e5d-b055-1781346c7462 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:53.719424 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1406/2825] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 23:02:53 2022] PUT /identity/v3/projects/fe9673034438436880d63ac5d3222b42/users/b49dd21d74454a56a0e84562160ef20b/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:53.732545 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a474af07-9fa7-46e6-9c25-7e9972e8090b None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:53.732834 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a474af07-9fa7-46e6-9c25-7e9972e8090b None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:53.733115 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a474af07-9fa7-46e6-9c25-7e9972e8090b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:53.761013 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-a474af07-9fa7-46e6-9c25-7e9972e8090b None None] MFA Rules not processed for user `b49dd21d74454a56a0e84562160ef20b`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:02:53.762224 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-a474af07-9fa7-46e6-9c25-7e9972e8090b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:53.833735 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1420/2826] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:02:53 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:02:53.845380 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e889f-8f60-4e5d-b055-1781346c7462 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:53.847184 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-c36e889f-8f60-4e5d-b055-1781346c7462 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:53.908908 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e889f-8f60-4e5d-b055-1781346c7462 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:53.912563 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83966bc0-85ae-48e7-abea-0331d236849d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:53.913209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83966bc0-85ae-48e7-abea-0331d236849d service cinder] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:53.913496 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83966bc0-85ae-48e7-abea-0331d236849d service cinder] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:53.913804 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83966bc0-85ae-48e7-abea-0331d236849d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:53.924179 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83966bc0-85ae-48e7-abea-0331d236849d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:53.925632 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83966bc0-85ae-48e7-abea-0331d236849d service cinder] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:53.935208 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1407/2827] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:02:53 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:02:54.064121 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a474af07-9fa7-46e6-9c25-7e9972e8090b None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:54.127688 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a474af07-9fa7-46e6-9c25-7e9972e8090b None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:54.132079 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a384263-b103-4587-9487-3201909772c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:54.132946 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a384263-b103-4587-9487-3201909772c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:54.133228 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a384263-b103-4587-9487-3201909772c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:54.134144 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a384263-b103-4587-9487-3201909772c1 admin admin] PATH_INFO: `/v3/users/b8533e19fbc5480291472bb103e656f3` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:54.135371 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a384263-b103-4587-9487-3201909772c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8533e19fbc5480291472bb103e656f3)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:54.137269 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a384263-b103-4587-9487-3201909772c1 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:54.316970 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-0a384263-b103-4587-9487-3201909772c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8533e19fbc5480291472bb103e656f3'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:54.324346 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-0a384263-b103-4587-9487-3201909772c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8533e19fbc5480291472bb103e656f3'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:54.325015 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-0a384263-b103-4587-9487-3201909772c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8533e19fbc5480291472bb103e656f3'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:54.330426 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:54.337308 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-0a384263-b103-4587-9487-3201909772c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8533e19fbc5480291472bb103e656f3'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:54.437355 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-0a384263-b103-4587-9487-3201909772c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8533e19fbc5480291472bb103e656f3'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:54.444084 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1421/2828] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 23:02:54 2022] DELETE /identity/v3/users/b8533e19fbc5480291472bb103e656f3 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:54.454434 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef364e6-90b8-4020-b3e2-e0c2ec2d44e0 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:54.454434 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef364e6-90b8-4020-b3e2-e0c2ec2d44e0 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:54.454434 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef364e6-90b8-4020-b3e2-e0c2ec2d44e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:54.466484 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-5ef364e6-90b8-4020-b3e2-e0c2ec2d44e0 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:02:54.467716 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-5ef364e6-90b8-4020-b3e2-e0c2ec2d44e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:54.534196 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1408/2829] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:02:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:02:54.543842 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a384263-b103-4587-9487-3201909772c1 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:54.545295 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-0a384263-b103-4587-9487-3201909772c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:54.603526 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a384263-b103-4587-9487-3201909772c1 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:54.607577 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166a90fa-5a9b-47c3-a765-d373988ee716 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:54.608216 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a90fa-5a9b-47c3-a765-d373988ee716 service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:54.608497 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a90fa-5a9b-47c3-a765-d373988ee716 service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:54.608770 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a90fa-5a9b-47c3-a765-d373988ee716 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:54.618361 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166a90fa-5a9b-47c3-a765-d373988ee716 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:54.619891 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166a90fa-5a9b-47c3-a765-d373988ee716 service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:54.630282 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1422/2830] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:02:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:02:55.035342 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef364e6-90b8-4020-b3e2-e0c2ec2d44e0 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:55.036766 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-5ef364e6-90b8-4020-b3e2-e0c2ec2d44e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:55.050624 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef364e6-90b8-4020-b3e2-e0c2ec2d44e0 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:55.056481 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951e738c-6efa-4f24-8558-c549c653d6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:55.057390 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e738c-6efa-4f24-8558-c549c653d6f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:55.057676 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e738c-6efa-4f24-8558-c549c653d6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:55.057957 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e738c-6efa-4f24-8558-c549c653d6f9 admin admin] PATH_INFO: `/v3/projects/ddf3224e839c48cb8a760afd3cb2c3b2` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:55.059194 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951e738c-6efa-4f24-8558-c549c653d6f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddf3224e839c48cb8a760afd3cb2c3b2)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:55.061103 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951e738c-6efa-4f24-8558-c549c653d6f9 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:55.163731 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-951e738c-6efa-4f24-8558-c549c653d6f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddf3224e839c48cb8a760afd3cb2c3b2', 'request_id': 'req-951e738c-6efa-4f24-8558-c549c653d6f9'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:55.170169 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-951e738c-6efa-4f24-8558-c549c653d6f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddf3224e839c48cb8a760afd3cb2c3b2', 'request_id': 'req-951e738c-6efa-4f24-8558-c549c653d6f9'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:55.172777 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:55.245229 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-951e738c-6efa-4f24-8558-c549c653d6f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddf3224e839c48cb8a760afd3cb2c3b2', 'request_id': 'req-951e738c-6efa-4f24-8558-c549c653d6f9'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:55.257200 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-951e738c-6efa-4f24-8558-c549c653d6f9 admin admin] The token cache is being invalidate because project ddf3224e839c48cb8a760afd3cb2c3b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:02:55.257508 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-951e738c-6efa-4f24-8558-c549c653d6f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:55.258152 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-951e738c-6efa-4f24-8558-c549c653d6f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:55.259947 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1409/2831] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 23:02:55 2022] DELETE /identity/v3/projects/ddf3224e839c48cb8a760afd3cb2c3b2 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:55.264018 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166a90fa-5a9b-47c3-a765-d373988ee716 service neutron] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:55.273984 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166a90fa-5a9b-47c3-a765-d373988ee716 service neutron] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:55.331499 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ab438f-36ff-47ef-bdf6-31cd02872417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:55.332324 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab438f-36ff-47ef-bdf6-31cd02872417 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:55.332608 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab438f-36ff-47ef-bdf6-31cd02872417 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:55.332885 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab438f-36ff-47ef-bdf6-31cd02872417 admin admin] PATH_INFO: `/v3/users/b49dd21d74454a56a0e84562160ef20b` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:55.334112 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ab438f-36ff-47ef-bdf6-31cd02872417 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b49dd21d74454a56a0e84562160ef20b)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:55.336092 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ab438f-36ff-47ef-bdf6-31cd02872417 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:55.521323 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-69ab438f-36ff-47ef-bdf6-31cd02872417 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b49dd21d74454a56a0e84562160ef20b'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:55.528446 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-69ab438f-36ff-47ef-bdf6-31cd02872417 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b49dd21d74454a56a0e84562160ef20b'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:55.533648 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-69ab438f-36ff-47ef-bdf6-31cd02872417 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b49dd21d74454a56a0e84562160ef20b'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:55.541217 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-69ab438f-36ff-47ef-bdf6-31cd02872417 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b49dd21d74454a56a0e84562160ef20b'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:55.603098 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-69ab438f-36ff-47ef-bdf6-31cd02872417 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b49dd21d74454a56a0e84562160ef20b'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:55.609799 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1423/2832] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 23:02:55 2022] DELETE /identity/v3/users/b49dd21d74454a56a0e84562160ef20b => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:56.039286 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951e738c-6efa-4f24-8558-c549c653d6f9 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:56.041296 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-951e738c-6efa-4f24-8558-c549c653d6f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:56.054258 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951e738c-6efa-4f24-8558-c549c653d6f9 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:56.113548 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5b532a-02e7-4842-a886-911cd4b219b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:56.114408 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b532a-02e7-4842-a886-911cd4b219b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:56.114693 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b532a-02e7-4842-a886-911cd4b219b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:56.114999 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b532a-02e7-4842-a886-911cd4b219b2 admin admin] PATH_INFO: `/v3/projects/fe9673034438436880d63ac5d3222b42` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:56.116170 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5b532a-02e7-4842-a886-911cd4b219b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe9673034438436880d63ac5d3222b42)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:56.118734 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5b532a-02e7-4842-a886-911cd4b219b2 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:56.233485 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-df5b532a-02e7-4842-a886-911cd4b219b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe9673034438436880d63ac5d3222b42', 'request_id': 'req-df5b532a-02e7-4842-a886-911cd4b219b2'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:56.240807 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-df5b532a-02e7-4842-a886-911cd4b219b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe9673034438436880d63ac5d3222b42', 'request_id': 'req-df5b532a-02e7-4842-a886-911cd4b219b2'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:56.319567 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-df5b532a-02e7-4842-a886-911cd4b219b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe9673034438436880d63ac5d3222b42', 'request_id': 'req-df5b532a-02e7-4842-a886-911cd4b219b2'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:56.331150 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-df5b532a-02e7-4842-a886-911cd4b219b2 admin admin] The token cache is being invalidate because project fe9673034438436880d63ac5d3222b42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:02:56.331567 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-df5b532a-02e7-4842-a886-911cd4b219b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:56.332181 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-df5b532a-02e7-4842-a886-911cd4b219b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:56.333700 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1410/2833] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 23:02:56 2022] DELETE /identity/v3/projects/fe9673034438436880d63ac5d3222b42 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:56.339071 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d293801-5c90-4f56-96ae-5b57d0248012 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:56.342837 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d293801-5c90-4f56-96ae-5b57d0248012 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:56.342837 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d293801-5c90-4f56-96ae-5b57d0248012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:56.346169 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:56.359261 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-3d293801-5c90-4f56-96ae-5b57d0248012 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:02:56.360605 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-3d293801-5c90-4f56-96ae-5b57d0248012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:56.420766 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1424/2834] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:02:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:02:56.428906 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6d7eba-33cc-441b-ab4f-4dee49908644 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:56.429209 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6d7eba-33cc-441b-ab4f-4dee49908644 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:56.429498 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6d7eba-33cc-441b-ab4f-4dee49908644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:56.447479 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-db6d7eba-33cc-441b-ab4f-4dee49908644 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:02:56.448686 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-db6d7eba-33cc-441b-ab4f-4dee49908644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:56.456626 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1411/2835] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:02:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:02:56.460829 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d293801-5c90-4f56-96ae-5b57d0248012 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:56.471117 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d293801-5c90-4f56-96ae-5b57d0248012 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:56.475661 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c1c29-49e9-49d7-bcbb-a43044e0d644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:56.476320 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c1c29-49e9-49d7-bcbb-a43044e0d644 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:56.476602 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c1c29-49e9-49d7-bcbb-a43044e0d644 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:56.476877 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c1c29-49e9-49d7-bcbb-a43044e0d644 admin admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:56.477603 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14c1c29-49e9-49d7-bcbb-a43044e0d644 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 23:02:56.477906 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14c1c29-49e9-49d7-bcbb-a43044e0d644 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:56.480520 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14c1c29-49e9-49d7-bcbb-a43044e0d644 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:56.492215 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1425/2836] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 23:02:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 23:02:56.496636 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6d7eba-33cc-441b-ab4f-4dee49908644 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:56.505893 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6d7eba-33cc-441b-ab4f-4dee49908644 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:56.511227 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa32f5ab-e49e-44d0-aac0-57c07c01d324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:56.511885 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa32f5ab-e49e-44d0-aac0-57c07c01d324 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:56.512168 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa32f5ab-e49e-44d0-aac0-57c07c01d324 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:56.512469 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa32f5ab-e49e-44d0-aac0-57c07c01d324 admin admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:56.513185 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa32f5ab-e49e-44d0-aac0-57c07c01d324 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:56.515112 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa32f5ab-e49e-44d0-aac0-57c07c01d324 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:56.617807 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1412/2837] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 23:02:56 2022] POST /identity/v3/projects => generated 383 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 23:02:56.622386 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c1c29-49e9-49d7-bcbb-a43044e0d644 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:56.636098 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c1c29-49e9-49d7-bcbb-a43044e0d644 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:56.692366 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa7652-6493-4b0b-af0f-432f28c7edfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:56.693035 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa7652-6493-4b0b-af0f-432f28c7edfa admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:56.693317 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa7652-6493-4b0b-af0f-432f28c7edfa admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:56.693593 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa7652-6493-4b0b-af0f-432f28c7edfa admin admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:56.694285 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaa7652-6493-4b0b-af0f-432f28c7edfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:56.696232 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaa7652-6493-4b0b-af0f-432f28c7edfa admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:56.805893 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1426/2838] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 23:02:56 2022] POST /identity/v3/users => generated 292 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:02:56.809949 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa32f5ab-e49e-44d0-aac0-57c07c01d324 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:56.820721 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa32f5ab-e49e-44d0-aac0-57c07c01d324 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:56.825756 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b20abf-ac7a-403e-930c-d5693bc4dcdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:56.826414 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b20abf-ac7a-403e-930c-d5693bc4dcdc admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:56.826717 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b20abf-ac7a-403e-930c-d5693bc4dcdc admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:56.827048 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b20abf-ac7a-403e-930c-d5693bc4dcdc admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:56.827862 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b20abf-ac7a-403e-930c-d5693bc4dcdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:56.829480 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b20abf-ac7a-403e-930c-d5693bc4dcdc admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:56.840633 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1413/2839] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:02:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:02:56.855302 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa7652-6493-4b0b-af0f-432f28c7edfa admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:56.865834 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa7652-6493-4b0b-af0f-432f28c7edfa admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:56.871086 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4638ed88-7f96-421e-b6cb-d7f13fcaa337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:56.871749 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4638ed88-7f96-421e-b6cb-d7f13fcaa337 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:56.872033 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4638ed88-7f96-421e-b6cb-d7f13fcaa337 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:56.872311 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4638ed88-7f96-421e-b6cb-d7f13fcaa337 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:56.873060 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4638ed88-7f96-421e-b6cb-d7f13fcaa337 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:56.874627 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4638ed88-7f96-421e-b6cb-d7f13fcaa337 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:56.885462 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1427/2840] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:02:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:02:56.889486 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b20abf-ac7a-403e-930c-d5693bc4dcdc admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:56.899520 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b20abf-ac7a-403e-930c-d5693bc4dcdc admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:56.904123 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ca125c-c15e-4c11-992c-9cb094c5bdf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:56.904995 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ca125c-c15e-4c11-992c-9cb094c5bdf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:56.905279 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ca125c-c15e-4c11-992c-9cb094c5bdf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:56.905557 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ca125c-c15e-4c11-992c-9cb094c5bdf8 admin admin] PATH_INFO: `/v3/projects/23cdf88fa83d4732b35a24fddae0e3a0/users/60b2abd7b4384edcb5108d69276f4aa2/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:56.919786 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ca125c-c15e-4c11-992c-9cb094c5bdf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23cdf88fa83d4732b35a24fddae0e3a0, user_id=60b2abd7b4384edcb5108d69276f4aa2, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:56.922461 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ca125c-c15e-4c11-992c-9cb094c5bdf8 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:57.005199 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1414/2841] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 23:02:56 2022] PUT /identity/v3/projects/23cdf88fa83d4732b35a24fddae0e3a0/users/60b2abd7b4384edcb5108d69276f4aa2/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:57.016624 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b412dee-1733-48a0-b34d-9fc76d06a691 None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:57.019339 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b412dee-1733-48a0-b34d-9fc76d06a691 None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:57.019339 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b412dee-1733-48a0-b34d-9fc76d06a691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:57.043611 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-6b412dee-1733-48a0-b34d-9fc76d06a691 None None] MFA Rules not processed for user `60b2abd7b4384edcb5108d69276f4aa2`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:02:57.044806 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-6b412dee-1733-48a0-b34d-9fc76d06a691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:57.102081 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:57.107698 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1428/2842] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:02:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:02:57.112131 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ca125c-c15e-4c11-992c-9cb094c5bdf8 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:57.130579 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:57.174659 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ca125c-c15e-4c11-992c-9cb094c5bdf8 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:57.179475 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b970fd-b956-401d-b994-2083915d9c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:57.180129 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b970fd-b956-401d-b994-2083915d9c6b admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:57.180409 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b970fd-b956-401d-b994-2083915d9c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:57.180680 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b970fd-b956-401d-b994-2083915d9c6b admin admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:57.181980 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b970fd-b956-401d-b994-2083915d9c6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:57.183949 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b970fd-b956-401d-b994-2083915d9c6b admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:57.309852 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1415/2843] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 23:02:57 2022] POST /identity/v3/projects => generated 381 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:02:57.313991 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b412dee-1733-48a0-b34d-9fc76d06a691 None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:57.324722 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b412dee-1733-48a0-b34d-9fc76d06a691 None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:57.386387 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2975547b-69a1-4577-ab63-be0789e9ba06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:57.387095 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975547b-69a1-4577-ab63-be0789e9ba06 admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:57.387303 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975547b-69a1-4577-ab63-be0789e9ba06 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:57.387583 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975547b-69a1-4577-ab63-be0789e9ba06 admin admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:57.388271 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2975547b-69a1-4577-ab63-be0789e9ba06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:57.390200 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2975547b-69a1-4577-ab63-be0789e9ba06 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:57.508294 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1429/2844] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 23:02:57 2022] POST /identity/v3/users => generated 297 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:02:57.512236 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b970fd-b956-401d-b994-2083915d9c6b admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:57.522560 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b970fd-b956-401d-b994-2083915d9c6b admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:57.527241 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8410f-5916-4972-b2ec-77f3e4f1f4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:57.527967 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8410f-5916-4972-b2ec-77f3e4f1f4c6 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:57.528235 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8410f-5916-4972-b2ec-77f3e4f1f4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:57.528516 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8410f-5916-4972-b2ec-77f3e4f1f4c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:57.529245 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd8410f-5916-4972-b2ec-77f3e4f1f4c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:57.531585 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd8410f-5916-4972-b2ec-77f3e4f1f4c6 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:57.542101 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1416/2845] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:02:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 23:02:57.546098 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2975547b-69a1-4577-ab63-be0789e9ba06 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:57.556129 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2975547b-69a1-4577-ab63-be0789e9ba06 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:57.561290 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04acdbcf-955c-490d-9104-dcda35ffcaa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:57.562157 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04acdbcf-955c-490d-9104-dcda35ffcaa3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:57.562442 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04acdbcf-955c-490d-9104-dcda35ffcaa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:57.562720 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04acdbcf-955c-490d-9104-dcda35ffcaa3 admin admin] PATH_INFO: `/v3/projects/6c6d355965fc46eb8413fe8e1a8fcb26/users/35712d872d324385a5776031bffc1110/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:57.578621 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04acdbcf-955c-490d-9104-dcda35ffcaa3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c6d355965fc46eb8413fe8e1a8fcb26, user_id=35712d872d324385a5776031bffc1110, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:57.581317 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04acdbcf-955c-490d-9104-dcda35ffcaa3 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:57.655050 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1430/2846] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 23:02:57 2022] PUT /identity/v3/projects/6c6d355965fc46eb8413fe8e1a8fcb26/users/35712d872d324385a5776031bffc1110/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:57.658494 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8410f-5916-4972-b2ec-77f3e4f1f4c6 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:57.671824 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8410f-5916-4972-b2ec-77f3e4f1f4c6 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:57.727555 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3059794-09c6-40c0-99e8-5d6b2d141a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:57.728846 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3059794-09c6-40c0-99e8-5d6b2d141a93 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:57.729137 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3059794-09c6-40c0-99e8-5d6b2d141a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:57.729443 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3059794-09c6-40c0-99e8-5d6b2d141a93 admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:57.730419 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3059794-09c6-40c0-99e8-5d6b2d141a93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:57.732038 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3059794-09c6-40c0-99e8-5d6b2d141a93 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:57.741580 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1417/2847] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:02:57 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:02:57.745701 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04acdbcf-955c-490d-9104-dcda35ffcaa3 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:57.756235 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04acdbcf-955c-490d-9104-dcda35ffcaa3 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:57.760818 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e507c-24b0-4957-a403-e701a6cc0464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:57.761727 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e507c-24b0-4957-a403-e701a6cc0464 admin admin] REQUEST_METHOD: `PUT` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:57.762014 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e507c-24b0-4957-a403-e701a6cc0464 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:57.762294 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e507c-24b0-4957-a403-e701a6cc0464 admin admin] PATH_INFO: `/v3/projects/6c6d355965fc46eb8413fe8e1a8fcb26/users/35712d872d324385a5776031bffc1110/roles/d1ddee590a964ff1bbb4ee8716173021` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:57.764335 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440e507c-24b0-4957-a403-e701a6cc0464 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c6d355965fc46eb8413fe8e1a8fcb26, user_id=35712d872d324385a5776031bffc1110, role_id=d1ddee590a964ff1bbb4ee8716173021)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:57.766977 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440e507c-24b0-4957-a403-e701a6cc0464 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:57.777652 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1431/2848] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 23:02:57 2022] PUT /identity/v3/projects/6c6d355965fc46eb8413fe8e1a8fcb26/users/35712d872d324385a5776031bffc1110/roles/d1ddee590a964ff1bbb4ee8716173021 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:57.788489 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed67a64-80e3-4154-a91d-fbddf56a4bd5 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:57.788782 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed67a64-80e3-4154-a91d-fbddf56a4bd5 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:57.789065 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed67a64-80e3-4154-a91d-fbddf56a4bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:57.818271 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-8ed67a64-80e3-4154-a91d-fbddf56a4bd5 None None] MFA Rules not processed for user `35712d872d324385a5776031bffc1110`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:02:57.819527 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-8ed67a64-80e3-4154-a91d-fbddf56a4bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:58.008020 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1418/2849] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:02:57 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:02:58.020189 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e507c-24b0-4957-a403-e701a6cc0464 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:58.022276 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-440e507c-24b0-4957-a403-e701a6cc0464 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:58.081622 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e507c-24b0-4957-a403-e701a6cc0464 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:58.085293 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d02e1be-165e-4403-a43e-c738b0c3c634 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:58.085938 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02e1be-165e-4403-a43e-c738b0c3c634 service cinder] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:58.086221 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02e1be-165e-4403-a43e-c738b0c3c634 service cinder] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:58.086505 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02e1be-165e-4403-a43e-c738b0c3c634 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:58.096285 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d02e1be-165e-4403-a43e-c738b0c3c634 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:58.097704 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d02e1be-165e-4403-a43e-c738b0c3c634 service cinder] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:58.107494 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1432/2850] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:02:58 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:02:58.241379 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed67a64-80e3-4154-a91d-fbddf56a4bd5 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:58.251085 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed67a64-80e3-4154-a91d-fbddf56a4bd5 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:58.255037 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2149b008-a8e0-40b4-8e91-eabfbeefc55d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:58.255378 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2149b008-a8e0-40b4-8e91-eabfbeefc55d service cinder] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:58.255662 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2149b008-a8e0-40b4-8e91-eabfbeefc55d service cinder] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:58.255938 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2149b008-a8e0-40b4-8e91-eabfbeefc55d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:58.264980 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:58.313654 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2149b008-a8e0-40b4-8e91-eabfbeefc55d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:58.315155 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2149b008-a8e0-40b4-8e91-eabfbeefc55d service cinder] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:58.325326 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1419/2851] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:02:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:02:58.814363 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d02e1be-165e-4403-a43e-c738b0c3c634 service cinder] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:58.865798 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:02:58.877019 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d02e1be-165e-4403-a43e-c738b0c3c634 service cinder] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:58.881685 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1b3537-c229-4661-858f-cd76d0c5f59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:58.882611 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1b3537-c229-4661-858f-cd76d0c5f59f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:58.882926 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1b3537-c229-4661-858f-cd76d0c5f59f admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:58.883224 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1b3537-c229-4661-858f-cd76d0c5f59f admin admin] PATH_INFO: `/v3/users/60b2abd7b4384edcb5108d69276f4aa2` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:58.884487 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1b3537-c229-4661-858f-cd76d0c5f59f admin admin] RBAC: Authorizing `identity:delete_user(user_id=60b2abd7b4384edcb5108d69276f4aa2)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:58.886394 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1b3537-c229-4661-858f-cd76d0c5f59f admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:59.070263 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-ec1b3537-c229-4661-858f-cd76d0c5f59f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60b2abd7b4384edcb5108d69276f4aa2'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:59.077057 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-ec1b3537-c229-4661-858f-cd76d0c5f59f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60b2abd7b4384edcb5108d69276f4aa2'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:59.077722 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-ec1b3537-c229-4661-858f-cd76d0c5f59f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60b2abd7b4384edcb5108d69276f4aa2'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:59.090431 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-ec1b3537-c229-4661-858f-cd76d0c5f59f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60b2abd7b4384edcb5108d69276f4aa2'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:59.159233 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-ec1b3537-c229-4661-858f-cd76d0c5f59f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60b2abd7b4384edcb5108d69276f4aa2'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:59.166055 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1433/2852] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 23:02:58 2022] DELETE /identity/v3/users/60b2abd7b4384edcb5108d69276f4aa2 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:59.170573 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e659e80d-b967-4132-ab38-f68bf77be8b2 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:59.170885 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e659e80d-b967-4132-ab38-f68bf77be8b2 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:59.171167 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e659e80d-b967-4132-ab38-f68bf77be8b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:59.188128 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-e659e80d-b967-4132-ab38-f68bf77be8b2 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:02:59.192612 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-e659e80d-b967-4132-ab38-f68bf77be8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:59.247629 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1420/2853] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:02:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:02:59.257252 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1b3537-c229-4661-858f-cd76d0c5f59f admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:59.258645 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-ec1b3537-c229-4661-858f-cd76d0c5f59f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:59.316196 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1b3537-c229-4661-858f-cd76d0c5f59f admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:59.319828 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7edcb-7fb7-4878-9dbe-19a76a431a6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:59.320555 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7edcb-7fb7-4878-9dbe-19a76a431a6c service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:59.320783 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7edcb-7fb7-4878-9dbe-19a76a431a6c service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:59.321057 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7edcb-7fb7-4878-9dbe-19a76a431a6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:59.331305 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada7edcb-7fb7-4878-9dbe-19a76a431a6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:59.332735 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada7edcb-7fb7-4878-9dbe-19a76a431a6c service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:59.342897 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1434/2854] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:02:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:02:59.743968 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e659e80d-b967-4132-ab38-f68bf77be8b2 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:59.745529 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-e659e80d-b967-4132-ab38-f68bf77be8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:02:59.759738 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e659e80d-b967-4132-ab38-f68bf77be8b2 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:02:59.778400 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8792ebb9-7fb5-448a-ac51-bf0549fcac62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:02:59.779478 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792ebb9-7fb5-448a-ac51-bf0549fcac62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:02:59.779707 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792ebb9-7fb5-448a-ac51-bf0549fcac62 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:02:59.779988 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792ebb9-7fb5-448a-ac51-bf0549fcac62 admin admin] PATH_INFO: `/v3/projects/23cdf88fa83d4732b35a24fddae0e3a0` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:02:59.781632 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8792ebb9-7fb5-448a-ac51-bf0549fcac62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23cdf88fa83d4732b35a24fddae0e3a0)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:02:59.783544 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8792ebb9-7fb5-448a-ac51-bf0549fcac62 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:02:59.886385 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-8792ebb9-7fb5-448a-ac51-bf0549fcac62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23cdf88fa83d4732b35a24fddae0e3a0', 'request_id': 'req-8792ebb9-7fb5-448a-ac51-bf0549fcac62'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:59.891886 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-8792ebb9-7fb5-448a-ac51-bf0549fcac62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23cdf88fa83d4732b35a24fddae0e3a0', 'request_id': 'req-8792ebb9-7fb5-448a-ac51-bf0549fcac62'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:59.948313 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-8792ebb9-7fb5-448a-ac51-bf0549fcac62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23cdf88fa83d4732b35a24fddae0e3a0', 'request_id': 'req-8792ebb9-7fb5-448a-ac51-bf0549fcac62'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:59.959348 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-8792ebb9-7fb5-448a-ac51-bf0549fcac62 admin admin] The token cache is being invalidate because project 23cdf88fa83d4732b35a24fddae0e3a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:02:59.959629 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-8792ebb9-7fb5-448a-ac51-bf0549fcac62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:59.960231 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-8792ebb9-7fb5-448a-ac51-bf0549fcac62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:02:59.961729 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1421/2855] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 23:02:59 2022] DELETE /identity/v3/projects/23cdf88fa83d4732b35a24fddae0e3a0 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:02:59.965716 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7edcb-7fb7-4878-9dbe-19a76a431a6c service neutron] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:02:59.975222 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7edcb-7fb7-4878-9dbe-19a76a431a6c service neutron] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:03:00.031864 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1b00d0-2415-40d4-bd7f-fea78f22fa23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:03:00.032727 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b00d0-2415-40d4-bd7f-fea78f22fa23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:03:00.033006 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b00d0-2415-40d4-bd7f-fea78f22fa23 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:03:00.033279 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b00d0-2415-40d4-bd7f-fea78f22fa23 admin admin] PATH_INFO: `/v3/users/35712d872d324385a5776031bffc1110` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:03:00.034456 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1b00d0-2415-40d4-bd7f-fea78f22fa23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35712d872d324385a5776031bffc1110)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:03:00.036390 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1b00d0-2415-40d4-bd7f-fea78f22fa23 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:03:00.051246 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:03:00.212921 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-db1b00d0-2415-40d4-bd7f-fea78f22fa23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35712d872d324385a5776031bffc1110'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:03:00.219899 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-db1b00d0-2415-40d4-bd7f-fea78f22fa23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35712d872d324385a5776031bffc1110'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:03:00.220570 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-db1b00d0-2415-40d4-bd7f-fea78f22fa23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35712d872d324385a5776031bffc1110'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:03:00.233380 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-db1b00d0-2415-40d4-bd7f-fea78f22fa23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35712d872d324385a5776031bffc1110'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:03:00.319464 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-db1b00d0-2415-40d4-bd7f-fea78f22fa23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35712d872d324385a5776031bffc1110'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:03:00.325713 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1435/2856] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 23:02:59 2022] DELETE /identity/v3/users/35712d872d324385a5776031bffc1110 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 23:03:00.758441 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8792ebb9-7fb5-448a-ac51-bf0549fcac62 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:03:00.758441 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-8792ebb9-7fb5-448a-ac51-bf0549fcac62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:03:00.771040 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8792ebb9-7fb5-448a-ac51-bf0549fcac62 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:03:00.805164 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:03:00.839695 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f6888a-4dcb-4068-868a-417fd3632d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:03:00.839695 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f6888a-4dcb-4068-868a-417fd3632d50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:03:00.839695 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f6888a-4dcb-4068-868a-417fd3632d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:03:00.839695 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f6888a-4dcb-4068-868a-417fd3632d50 admin admin] PATH_INFO: `/v3/projects/6c6d355965fc46eb8413fe8e1a8fcb26` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:03:00.840214 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f6888a-4dcb-4068-868a-417fd3632d50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c6d355965fc46eb8413fe8e1a8fcb26)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:03:00.840214 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f6888a-4dcb-4068-868a-417fd3632d50 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:03:00.971772 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-c0f6888a-4dcb-4068-868a-417fd3632d50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c6d355965fc46eb8413fe8e1a8fcb26', 'request_id': 'req-c0f6888a-4dcb-4068-868a-417fd3632d50'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:03:00.977357 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-c0f6888a-4dcb-4068-868a-417fd3632d50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c6d355965fc46eb8413fe8e1a8fcb26', 'request_id': 'req-c0f6888a-4dcb-4068-868a-417fd3632d50'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:03:01.056793 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-c0f6888a-4dcb-4068-868a-417fd3632d50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c6d355965fc46eb8413fe8e1a8fcb26', 'request_id': 'req-c0f6888a-4dcb-4068-868a-417fd3632d50'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:03:01.068051 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-c0f6888a-4dcb-4068-868a-417fd3632d50 admin admin] The token cache is being invalidate because project 6c6d355965fc46eb8413fe8e1a8fcb26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:03:01.068345 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-c0f6888a-4dcb-4068-868a-417fd3632d50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:03:01.068976 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-c0f6888a-4dcb-4068-868a-417fd3632d50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:03:01.070540 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1422/2857] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 23:03:00 2022] DELETE /identity/v3/projects/6c6d355965fc46eb8413fe8e1a8fcb26 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:03:01.076134 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b3a1b3-55eb-4fe6-9f0d-224e47374e1b None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:03:01.076618 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b3a1b3-55eb-4fe6-9f0d-224e47374e1b None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:03:01.076744 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b3a1b3-55eb-4fe6-9f0d-224e47374e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:03:01.095598 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-a9b3a1b3-55eb-4fe6-9f0d-224e47374e1b None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:03:01.096789 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-a9b3a1b3-55eb-4fe6-9f0d-224e47374e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:03:01.156236 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1436/2858] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:03:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:03:01.164284 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb942c6-f852-461c-be3f-60d1f2cf602e None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:03:01.164577 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb942c6-f852-461c-be3f-60d1f2cf602e None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:03:01.164861 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb942c6-f852-461c-be3f-60d1f2cf602e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:03:01.182901 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-4bb942c6-f852-461c-be3f-60d1f2cf602e None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:03:01.184174 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-4bb942c6-f852-461c-be3f-60d1f2cf602e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:03:01.192661 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1423/2859] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:03:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:03:01.197482 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b3a1b3-55eb-4fe6-9f0d-224e47374e1b None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:03:01.207261 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b3a1b3-55eb-4fe6-9f0d-224e47374e1b None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:03:01.211860 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8df1705-c025-4fc7-b18a-60ee9221ccf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:03:01.213173 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df1705-c025-4fc7-b18a-60ee9221ccf1 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:03:01.213458 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df1705-c025-4fc7-b18a-60ee9221ccf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:03:01.213732 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df1705-c025-4fc7-b18a-60ee9221ccf1 admin admin] PATH_INFO: `/v3/domains` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:03:01.214451 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8df1705-c025-4fc7-b18a-60ee9221ccf1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 23:03:01.214756 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8df1705-c025-4fc7-b18a-60ee9221ccf1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:03:01.216368 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8df1705-c025-4fc7-b18a-60ee9221ccf1 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:03:01.228138 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1437/2860] 10.222.0.52 () {58 vars in 1152 bytes} [Thu Jul 21 23:03:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 23:03:01.232714 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb942c6-f852-461c-be3f-60d1f2cf602e None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:03:01.242082 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb942c6-f852-461c-be3f-60d1f2cf602e None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:03:01.246647 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31318a81-79cf-485b-943b-59088ef49583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:03:01.247479 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31318a81-79cf-485b-943b-59088ef49583 admin admin] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:03:01.247768 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31318a81-79cf-485b-943b-59088ef49583 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:03:01.248044 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31318a81-79cf-485b-943b-59088ef49583 admin admin] PATH_INFO: `/v3/projects` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:03:01.248732 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31318a81-79cf-485b-943b-59088ef49583 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:03:01.250592 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31318a81-79cf-485b-943b-59088ef49583 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:03:01.338602 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1424/2861] 10.222.0.52 () {60 vars in 1152 bytes} [Thu Jul 21 23:03:01 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 23:03:01.342964 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8df1705-c025-4fc7-b18a-60ee9221ccf1 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:03:01.352969 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8df1705-c025-4fc7-b18a-60ee9221ccf1 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:03:01.411130 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721adef8-47a0-416e-b448-99909a07490f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:03:01.411802 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721adef8-47a0-416e-b448-99909a07490f admin admin] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:03:01.412090 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721adef8-47a0-416e-b448-99909a07490f admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:03:01.412370 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721adef8-47a0-416e-b448-99909a07490f admin admin] PATH_INFO: `/v3/users` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:03:01.413055 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721adef8-47a0-416e-b448-99909a07490f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:03:01.415041 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721adef8-47a0-416e-b448-99909a07490f admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:03:01.502833 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1438/2862] 10.222.0.52 () {60 vars in 1143 bytes} [Thu Jul 21 23:03:01 2022] POST /identity/v3/users => generated 287 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 23:03:01.507116 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31318a81-79cf-485b-943b-59088ef49583 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:03:01.518416 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31318a81-79cf-485b-943b-59088ef49583 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:03:01.523003 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dbd3d1-bfb7-4bd5-bced-5396b22015b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:03:01.523720 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dbd3d1-bfb7-4bd5-bced-5396b22015b8 admin admin] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:03:01.524020 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dbd3d1-bfb7-4bd5-bced-5396b22015b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:03:01.524294 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dbd3d1-bfb7-4bd5-bced-5396b22015b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:03:01.525042 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dbd3d1-bfb7-4bd5-bced-5396b22015b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:03:01.526636 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dbd3d1-bfb7-4bd5-bced-5396b22015b8 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:03:01.537169 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1425/2863] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:03:01.541106 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721adef8-47a0-416e-b448-99909a07490f admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:03:01.551291 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721adef8-47a0-416e-b448-99909a07490f admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:03:01.555837 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf09d7b3-44f3-424f-aea0-573adf67c323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:03:01.557058 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf09d7b3-44f3-424f-aea0-573adf67c323 admin admin] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:03:01.557342 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf09d7b3-44f3-424f-aea0-573adf67c323 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:03:01.557620 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf09d7b3-44f3-424f-aea0-573adf67c323 admin admin] PATH_INFO: `/v3/roles` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:03:01.558344 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf09d7b3-44f3-424f-aea0-573adf67c323 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:03:01.560007 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf09d7b3-44f3-424f-aea0-573adf67c323 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:03:01.565209 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:03:01.571056 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1439/2864] 10.222.0.52 () {58 vars in 1121 bytes} [Thu Jul 21 23:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 23:03:01.575619 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dbd3d1-bfb7-4bd5-bced-5396b22015b8 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:03:01.585629 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dbd3d1-bfb7-4bd5-bced-5396b22015b8 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:03:01.590274 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e5ce92-f764-48f5-af44-61800ef3311d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:03:01.591162 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e5ce92-f764-48f5-af44-61800ef3311d admin admin] REQUEST_METHOD: `PUT` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:03:01.591445 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e5ce92-f764-48f5-af44-61800ef3311d admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:03:01.591719 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e5ce92-f764-48f5-af44-61800ef3311d admin admin] PATH_INFO: `/v3/projects/3e246f912dba418bbaed9df7ade44809/users/18484d026cc2467fab1ac36f419b5c77/roles/53d3bdbe9cdf4d35bc73854a33df8e45` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:03:01.607063 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e5ce92-f764-48f5-af44-61800ef3311d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e246f912dba418bbaed9df7ade44809, user_id=18484d026cc2467fab1ac36f419b5c77, role_id=53d3bdbe9cdf4d35bc73854a33df8e45)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:03:01.609666 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e5ce92-f764-48f5-af44-61800ef3311d admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:03:01.673724 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1426/2865] 10.222.0.52 () {60 vars in 1482 bytes} [Thu Jul 21 23:03:01 2022] PUT /identity/v3/projects/3e246f912dba418bbaed9df7ade44809/users/18484d026cc2467fab1ac36f419b5c77/roles/53d3bdbe9cdf4d35bc73854a33df8e45 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:03:01.684886 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ff648-7346-4815-9fe7-6b6c00d0d08d None None] REQUEST_METHOD: `POST` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:03:01.685164 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ff648-7346-4815-9fe7-6b6c00d0d08d None None] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:03:01.685437 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ff648-7346-4815-9fe7-6b6c00d0d08d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:03:01.712405 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.auth.core [None req-6c3ff648-7346-4815-9fe7-6b6c00d0d08d None None] MFA Rules not processed for user `18484d026cc2467fab1ac36f419b5c77`. Rule list: `[]` (Enabled: `True`). {{(pid=87167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:03:01.713756 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-6c3ff648-7346-4815-9fe7-6b6c00d0d08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:03:01.778658 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1440/2866] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:03:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:03:01.789599 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e5ce92-f764-48f5-af44-61800ef3311d admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:03:01.791885 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-d8e5ce92-f764-48f5-af44-61800ef3311d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:03:01.847506 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e5ce92-f764-48f5-af44-61800ef3311d admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:03:01.851104 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5767240-1b22-489f-b62f-540a5d931544 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:03:01.851745 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5767240-1b22-489f-b62f-540a5d931544 service cinder] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:03:01.852029 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5767240-1b22-489f-b62f-540a5d931544 service cinder] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:03:01.852306 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5767240-1b22-489f-b62f-540a5d931544 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:03:01.862235 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5767240-1b22-489f-b62f-540a5d931544 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:03:01.867175 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5767240-1b22-489f-b62f-540a5d931544 service cinder] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:03:01.873933 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1427/2867] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:03:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:03:01.894100 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3ff648-7346-4815-9fe7-6b6c00d0d08d None None] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:03:01.962063 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3ff648-7346-4815-9fe7-6b6c00d0d08d None None] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:03:01.966694 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dfe9b9-9d12-43d2-ba95-69915b79e7d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:03:01.967605 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dfe9b9-9d12-43d2-ba95-69915b79e7d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:03:01.967893 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dfe9b9-9d12-43d2-ba95-69915b79e7d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:03:01.968169 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dfe9b9-9d12-43d2-ba95-69915b79e7d3 admin admin] PATH_INFO: `/v3/users/18484d026cc2467fab1ac36f419b5c77` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:03:01.969477 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dfe9b9-9d12-43d2-ba95-69915b79e7d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18484d026cc2467fab1ac36f419b5c77)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:03:01.971421 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dfe9b9-9d12-43d2-ba95-69915b79e7d3 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:03:02.185094 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-15dfe9b9-9d12-43d2-ba95-69915b79e7d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18484d026cc2467fab1ac36f419b5c77'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:03:02.192149 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-15dfe9b9-9d12-43d2-ba95-69915b79e7d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18484d026cc2467fab1ac36f419b5c77'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:03:02.192893 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-15dfe9b9-9d12-43d2-ba95-69915b79e7d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18484d026cc2467fab1ac36f419b5c77'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:03:02.205405 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-15dfe9b9-9d12-43d2-ba95-69915b79e7d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18484d026cc2467fab1ac36f419b5c77'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:03:02.306292 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-15dfe9b9-9d12-43d2-ba95-69915b79e7d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18484d026cc2467fab1ac36f419b5c77'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:03:02.313316 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1441/2868] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 23:03:01 2022] DELETE /identity/v3/users/18484d026cc2467fab1ac36f419b5c77 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:03:02.317914 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff808f6-2d64-4532-8761-4a2d7c044b74 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:03:02.318245 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff808f6-2d64-4532-8761-4a2d7c044b74 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:03:02.318504 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff808f6-2d64-4532-8761-4a2d7c044b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:03:02.324359 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:03:02.336917 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-9ff808f6-2d64-4532-8761-4a2d7c044b74 None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:03:02.340675 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-9ff808f6-2d64-4532-8761-4a2d7c044b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:03:02.396988 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1428/2869] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:03:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 23:03:02.407857 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dfe9b9-9d12-43d2-ba95-69915b79e7d3 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:03:02.409255 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-15dfe9b9-9d12-43d2-ba95-69915b79e7d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:03:02.421442 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:03:02.467790 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dfe9b9-9d12-43d2-ba95-69915b79e7d3 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:03:02.471434 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c5bee-a81f-4c91-8969-0ff545f6a9e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:03:02.472078 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c5bee-a81f-4c91-8969-0ff545f6a9e2 service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:03:02.472360 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c5bee-a81f-4c91-8969-0ff545f6a9e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:03:02.472638 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c5bee-a81f-4c91-8969-0ff545f6a9e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:03:02.482980 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7c5bee-a81f-4c91-8969-0ff545f6a9e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:03:02.484419 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7c5bee-a81f-4c91-8969-0ff545f6a9e2 service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:03:02.494064 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1442/2870] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:03:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:03:02.910132 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff808f6-2d64-4532-8761-4a2d7c044b74 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:03:02.910132 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-9ff808f6-2d64-4532-8761-4a2d7c044b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:03:02.917671 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff808f6-2d64-4532-8761-4a2d7c044b74 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:03:02.922606 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aba8e9-df41-42e2-ac13-b81c37175586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:03:02.923672 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aba8e9-df41-42e2-ac13-b81c37175586 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:03:02.923962 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aba8e9-df41-42e2-ac13-b81c37175586 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:03:02.924253 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aba8e9-df41-42e2-ac13-b81c37175586 admin admin] PATH_INFO: `/v3/projects/3e246f912dba418bbaed9df7ade44809` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:03:02.925694 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aba8e9-df41-42e2-ac13-b81c37175586 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e246f912dba418bbaed9df7ade44809)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:03:02.927849 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aba8e9-df41-42e2-ac13-b81c37175586 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:03:03.049675 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-62aba8e9-df41-42e2-ac13-b81c37175586 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e246f912dba418bbaed9df7ade44809', 'request_id': 'req-62aba8e9-df41-42e2-ac13-b81c37175586'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:03:03.055264 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-62aba8e9-df41-42e2-ac13-b81c37175586 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e246f912dba418bbaed9df7ade44809', 'request_id': 'req-62aba8e9-df41-42e2-ac13-b81c37175586'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:03:03.142698 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-62aba8e9-df41-42e2-ac13-b81c37175586 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e246f912dba418bbaed9df7ade44809', 'request_id': 'req-62aba8e9-df41-42e2-ac13-b81c37175586'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:03:03.154389 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-62aba8e9-df41-42e2-ac13-b81c37175586 admin admin] The token cache is being invalidate because project 3e246f912dba418bbaed9df7ade44809 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:03:03.154675 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-62aba8e9-df41-42e2-ac13-b81c37175586 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:03:03.155322 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-62aba8e9-df41-42e2-ac13-b81c37175586 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:03:03.156842 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1429/2871] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 23:03:02 2022] DELETE /identity/v3/projects/3e246f912dba418bbaed9df7ade44809 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:03:07.710865 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c5bee-a81f-4c91-8969-0ff545f6a9e2 service neutron] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:03:07.712384 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-3b7c5bee-a81f-4c91-8969-0ff545f6a9e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:03:07.725279 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c5bee-a81f-4c91-8969-0ff545f6a9e2 service neutron] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:03:07.780815 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b1691-2cea-4f65-862b-a6ade97d14e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:03:07.781607 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b1691-2cea-4f65-862b-a6ade97d14e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:03:07.781887 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b1691-2cea-4f65-862b-a6ade97d14e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:03:07.782160 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b1691-2cea-4f65-862b-a6ade97d14e9 admin admin] PATH_INFO: `/v3/users/238eb2d2b5c44d419a5d41267bd4dce0` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:03:07.783383 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1b1691-2cea-4f65-862b-a6ade97d14e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=238eb2d2b5c44d419a5d41267bd4dce0)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:03:07.785265 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1b1691-2cea-4f65-862b-a6ade97d14e9 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:03:07.950214 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-3b1b1691-2cea-4f65-862b-a6ade97d14e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '238eb2d2b5c44d419a5d41267bd4dce0'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:03:07.957225 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-3b1b1691-2cea-4f65-862b-a6ade97d14e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '238eb2d2b5c44d419a5d41267bd4dce0'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:03:07.957899 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-3b1b1691-2cea-4f65-862b-a6ade97d14e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '238eb2d2b5c44d419a5d41267bd4dce0'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:03:07.969638 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-3b1b1691-2cea-4f65-862b-a6ade97d14e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '238eb2d2b5c44d419a5d41267bd4dce0'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:03:08.050910 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-3b1b1691-2cea-4f65-862b-a6ade97d14e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '238eb2d2b5c44d419a5d41267bd4dce0'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:03:08.057616 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1443/2872] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 23:03:07 2022] DELETE /identity/v3/users/238eb2d2b5c44d419a5d41267bd4dce0 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:03:08.062086 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5990cc1-f738-47f3-8363-1ff4c6e2eb1f None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:03:08.062542 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5990cc1-f738-47f3-8363-1ff4c6e2eb1f None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:03:08.062674 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5990cc1-f738-47f3-8363-1ff4c6e2eb1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:03:08.079894 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-a5990cc1-f738-47f3-8363-1ff4c6e2eb1f None None] MFA Rules not processed for user `031023a71a014731ad7c782eb926f065`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:03:08.081111 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-a5990cc1-f738-47f3-8363-1ff4c6e2eb1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:03:08.139288 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1430/2873] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:03:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:03:08.148845 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b1691-2cea-4f65-862b-a6ade97d14e9 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:03:08.150216 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-3b1b1691-2cea-4f65-862b-a6ade97d14e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:03:08.185609 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:03:08.207747 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b1691-2cea-4f65-862b-a6ade97d14e9 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:03:08.211602 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0789a765-7ea5-4aab-8a33-ab19dad535c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:03:08.212253 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0789a765-7ea5-4aab-8a33-ab19dad535c9 service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:03:08.212540 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0789a765-7ea5-4aab-8a33-ab19dad535c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:03:08.212821 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0789a765-7ea5-4aab-8a33-ab19dad535c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:03:08.222995 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0789a765-7ea5-4aab-8a33-ab19dad535c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:03:08.224426 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0789a765-7ea5-4aab-8a33-ab19dad535c9 service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:03:08.234734 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1444/2874] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:03:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:03:08.623744 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5990cc1-f738-47f3-8363-1ff4c6e2eb1f None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:03:08.625275 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-a5990cc1-f738-47f3-8363-1ff4c6e2eb1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:03:08.638519 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5990cc1-f738-47f3-8363-1ff4c6e2eb1f None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:03:08.643237 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72783a7e-adc9-4388-b645-66d4042834a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:03:08.644162 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72783a7e-adc9-4388-b645-66d4042834a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:03:08.645039 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72783a7e-adc9-4388-b645-66d4042834a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:03:08.645326 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72783a7e-adc9-4388-b645-66d4042834a1 admin admin] PATH_INFO: `/v3/projects/935079a6a1634e61a15eb5740c3cdfb9` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:03:08.646762 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72783a7e-adc9-4388-b645-66d4042834a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=935079a6a1634e61a15eb5740c3cdfb9)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:03:08.648808 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72783a7e-adc9-4388-b645-66d4042834a1 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:03:08.673291 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:03:08.741818 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-72783a7e-adc9-4388-b645-66d4042834a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '935079a6a1634e61a15eb5740c3cdfb9', 'request_id': 'req-72783a7e-adc9-4388-b645-66d4042834a1'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:03:08.747371 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-72783a7e-adc9-4388-b645-66d4042834a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '935079a6a1634e61a15eb5740c3cdfb9', 'request_id': 'req-72783a7e-adc9-4388-b645-66d4042834a1'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:03:08.814190 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-72783a7e-adc9-4388-b645-66d4042834a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '935079a6a1634e61a15eb5740c3cdfb9', 'request_id': 'req-72783a7e-adc9-4388-b645-66d4042834a1'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:03:08.825979 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-72783a7e-adc9-4388-b645-66d4042834a1 admin admin] The token cache is being invalidate because project 935079a6a1634e61a15eb5740c3cdfb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:03:08.826289 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-72783a7e-adc9-4388-b645-66d4042834a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:03:08.827063 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-72783a7e-adc9-4388-b645-66d4042834a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:03:08.829241 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1431/2875] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 23:03:08 2022] DELETE /identity/v3/projects/935079a6a1634e61a15eb5740c3cdfb9 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 23:03:18.279224 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=12 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:03:18.828477 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:DELETE=7,keystone:SELECT=15,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:04:21.066409 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0789a765-7ea5-4aab-8a33-ab19dad535c9 service neutron] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:04:21.075975 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0789a765-7ea5-4aab-8a33-ab19dad535c9 service neutron] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:04:21.124274 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76481747-d178-48c8-98dc-1337cc332783 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:04:21.124917 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76481747-d178-48c8-98dc-1337cc332783 service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:04:21.125201 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76481747-d178-48c8-98dc-1337cc332783 service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:04:21.125480 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76481747-d178-48c8-98dc-1337cc332783 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:04:21.127302 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-76481747-d178-48c8-98dc-1337cc332783 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:04:21.196237 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76481747-d178-48c8-98dc-1337cc332783 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:04:21.197678 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76481747-d178-48c8-98dc-1337cc332783 service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:04:21.208506 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1445/2876] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:04:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:04:22.390277 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72783a7e-adc9-4388-b645-66d4042834a1 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:04:22.391798 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-72783a7e-adc9-4388-b645-66d4042834a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:04:22.403943 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72783a7e-adc9-4388-b645-66d4042834a1 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:04:22.407486 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9365d034-e265-436b-92dd-36158c384619 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:04:22.408124 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9365d034-e265-436b-92dd-36158c384619 service neutron] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:04:22.408411 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9365d034-e265-436b-92dd-36158c384619 service neutron] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:04:22.408696 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9365d034-e265-436b-92dd-36158c384619 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:04:22.410410 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-9365d034-e265-436b-92dd-36158c384619 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:04:22.423685 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9365d034-e265-436b-92dd-36158c384619 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:04:22.425040 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9365d034-e265-436b-92dd-36158c384619 service neutron] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:04:22.436525 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1432/2877] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:04:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:04:23.474910 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76481747-d178-48c8-98dc-1337cc332783 service neutron] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:04:23.476308 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-76481747-d178-48c8-98dc-1337cc332783 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:04:23.495584 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76481747-d178-48c8-98dc-1337cc332783 service neutron] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:04:23.501324 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c212e642-bedf-41fa-aa07-2af1ac05b5cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bfdb655a2a44b33ab724daaf73a294b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:04:23.502431 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212e642-bedf-41fa-aa07-2af1ac05b5cb service nova] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:04:23.502952 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212e642-bedf-41fa-aa07-2af1ac05b5cb service nova] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:04:23.503843 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212e642-bedf-41fa-aa07-2af1ac05b5cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:04:23.505994 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-c212e642-bedf-41fa-aa07-2af1ac05b5cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:04:23.521579 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c212e642-bedf-41fa-aa07-2af1ac05b5cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:04:23.523315 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c212e642-bedf-41fa-aa07-2af1ac05b5cb service nova] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:04:23.536308 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1446/2878] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:04:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:04:25.858941 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf5b43-dbb9-4417-bcb0-7179c8146376 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:04:25.859435 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf5b43-dbb9-4417-bcb0-7179c8146376 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:04:25.859540 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf5b43-dbb9-4417-bcb0-7179c8146376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:04:25.877343 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-a6bf5b43-dbb9-4417-bcb0-7179c8146376 None None] MFA Rules not processed for user `ae2a190cf2ef4777a3838cbe9750afea`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:04:25.878562 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-a6bf5b43-dbb9-4417-bcb0-7179c8146376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:04:25.931534 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1433/2879] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:04:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:04:26.236586 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c212e642-bedf-41fa-aa07-2af1ac05b5cb service nova] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:04:26.237989 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-c212e642-bedf-41fa-aa07-2af1ac05b5cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:04:26.249554 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c212e642-bedf-41fa-aa07-2af1ac05b5cb service nova] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:04:26.253078 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feef9aa-fb2f-4a47-9851-980d862a1324 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:04:26.253713 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feef9aa-fb2f-4a47-9851-980d862a1324 service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:04:26.253997 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feef9aa-fb2f-4a47-9851-980d862a1324 service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:04:26.254273 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feef9aa-fb2f-4a47-9851-980d862a1324 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:04:26.256022 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-7feef9aa-fb2f-4a47-9851-980d862a1324 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:04:26.265244 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feef9aa-fb2f-4a47-9851-980d862a1324 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:04:26.266832 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feef9aa-fb2f-4a47-9851-980d862a1324 service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:04:26.275445 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1447/2880] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:04:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:04:27.047943 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bf5b43-dbb9-4417-bcb0-7179c8146376 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:04:27.049496 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-a6bf5b43-dbb9-4417-bcb0-7179c8146376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:04:27.124756 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bf5b43-dbb9-4417-bcb0-7179c8146376 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:04:27.129669 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a180b9b-ec02-4293-8aee-34f739c67e65 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10c0bed169344a77bfad29624c5e5a8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:04:27.130310 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a180b9b-ec02-4293-8aee-34f739c67e65 service placement] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:04:27.130590 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a180b9b-ec02-4293-8aee-34f739c67e65 service placement] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:04:27.130893 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a180b9b-ec02-4293-8aee-34f739c67e65 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:04:27.132649 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-8a180b9b-ec02-4293-8aee-34f739c67e65 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:04:27.144668 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a180b9b-ec02-4293-8aee-34f739c67e65 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:04:27.146302 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a180b9b-ec02-4293-8aee-34f739c67e65 service placement] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:04:27.156471 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1434/2881] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:04:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:04:30.005834 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feef9aa-fb2f-4a47-9851-980d862a1324 service neutron] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:04:30.015223 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feef9aa-fb2f-4a47-9851-980d862a1324 service neutron] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:04:30.018974 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc26c133-4639-486c-bfae-10fd4a829c1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:04:30.019611 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc26c133-4639-486c-bfae-10fd4a829c1e service neutron] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:04:30.019892 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc26c133-4639-486c-bfae-10fd4a829c1e service neutron] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:04:30.020161 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc26c133-4639-486c-bfae-10fd4a829c1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:04:30.021905 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-dc26c133-4639-486c-bfae-10fd4a829c1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:04:30.032341 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc26c133-4639-486c-bfae-10fd4a829c1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:04:30.033765 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc26c133-4639-486c-bfae-10fd4a829c1e service neutron] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:04:30.042607 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1448/2882] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:04:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:04:37.152188 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=63 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:04:40.038193 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=72 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:04:43.260770 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a180b9b-ec02-4293-8aee-34f739c67e65 service placement] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:04:43.272807 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a180b9b-ec02-4293-8aee-34f739c67e65 service placement] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:04:43.279094 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ffbc57-3c79-4211-9a78-db9ef6ac181b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bfdb655a2a44b33ab724daaf73a294b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:04:43.282577 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ffbc57-3c79-4211-9a78-db9ef6ac181b service nova] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:04:43.282577 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ffbc57-3c79-4211-9a78-db9ef6ac181b service nova] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:04:43.282577 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ffbc57-3c79-4211-9a78-db9ef6ac181b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:04:43.282577 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-78ffbc57-3c79-4211-9a78-db9ef6ac181b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:04:43.294414 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ffbc57-3c79-4211-9a78-db9ef6ac181b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:04:43.296005 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ffbc57-3c79-4211-9a78-db9ef6ac181b service nova] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:04:43.306609 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1435/2883] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:04:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:04:53.302254 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=6 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:06:18.532368 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc26c133-4639-486c-bfae-10fd4a829c1e service neutron] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:06:18.533771 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-dc26c133-4639-486c-bfae-10fd4a829c1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:06:18.589603 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc26c133-4639-486c-bfae-10fd4a829c1e service neutron] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:06:18.593210 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d141d-c533-490b-ad4a-cd9b0d3d8d0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '150f42651f754fda83f25fe3b408bce9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:06:18.593850 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d141d-c533-490b-ad4a-cd9b0d3d8d0e service cinder] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:06:18.594137 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d141d-c533-490b-ad4a-cd9b0d3d8d0e service cinder] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:06:18.594417 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d141d-c533-490b-ad4a-cd9b0d3d8d0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:06:18.596191 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-892d141d-c533-490b-ad4a-cd9b0d3d8d0e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:06:18.607250 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892d141d-c533-490b-ad4a-cd9b0d3d8d0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:06:18.608668 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892d141d-c533-490b-ad4a-cd9b0d3d8d0e service cinder] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:06:18.617982 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1449/2884] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:06:18 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:06:23.508323 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6187533b-9455-4b97-950a-e588c3f90fa4 None None] REQUEST_METHOD: `POST` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:06:23.508787 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6187533b-9455-4b97-950a-e588c3f90fa4 None None] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:06:23.508919 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6187533b-9455-4b97-950a-e588c3f90fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:06:23.526728 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.auth.core [None req-6187533b-9455-4b97-950a-e588c3f90fa4 None None] MFA Rules not processed for user `ae2a190cf2ef4777a3838cbe9750afea`. Rule list: `[]` (Enabled: `True`). {{(pid=87168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 23:06:23.527936 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-6187533b-9455-4b97-950a-e588c3f90fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:06:23.534627 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1436/2885] 10.222.0.52 () {58 vars in 957 bytes} [Thu Jul 21 23:06:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 23:06:25.749040 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d141d-c533-490b-ad4a-cd9b0d3d8d0e service cinder] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:06:25.760819 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d141d-c533-490b-ad4a-cd9b0d3d8d0e service cinder] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:06:25.765948 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1811d4-8d21-4dd8-bbeb-1669d98a7777 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10c0bed169344a77bfad29624c5e5a8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:06:25.766585 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1811d4-8d21-4dd8-bbeb-1669d98a7777 service placement] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:06:25.766914 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1811d4-8d21-4dd8-bbeb-1669d98a7777 service placement] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:06:25.767197 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1811d4-8d21-4dd8-bbeb-1669d98a7777 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:06:25.768976 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-8e1811d4-8d21-4dd8-bbeb-1669d98a7777 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:06:25.781115 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1811d4-8d21-4dd8-bbeb-1669d98a7777 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:06:25.782791 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1811d4-8d21-4dd8-bbeb-1669d98a7777 service placement] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:06:25.793462 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1450/2886] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:06:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:06:33.521497 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=3 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:06:35.788639 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=32 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:08:29.540962 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6187533b-9455-4b97-950a-e588c3f90fa4 None None] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:08:29.550982 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6187533b-9455-4b97-950a-e588c3f90fa4 None None] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:08:29.554747 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca70a70-d5b3-4e12-bbd8-9507f69a8488 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48c3e33f24ff4d66a72f06848d769ac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:08:29.555417 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca70a70-d5b3-4e12-bbd8-9507f69a8488 service neutron] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:08:29.555706 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca70a70-d5b3-4e12-bbd8-9507f69a8488 service neutron] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:08:29.556488 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca70a70-d5b3-4e12-bbd8-9507f69a8488 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:08:29.558273 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-2ca70a70-d5b3-4e12-bbd8-9507f69a8488 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:08:29.623364 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca70a70-d5b3-4e12-bbd8-9507f69a8488 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:08:29.624791 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca70a70-d5b3-4e12-bbd8-9507f69a8488 service neutron] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:08:29.635044 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1437/2887] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:08:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 23:08:33.052021 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1811d4-8d21-4dd8-bbeb-1669d98a7777 service placement] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:08:33.063477 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1811d4-8d21-4dd8-bbeb-1669d98a7777 service placement] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:08:33.069414 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c478370-ec47-4eb8-bf41-b5bf7b8622d4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10c0bed169344a77bfad29624c5e5a8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:08:33.070209 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c478370-ec47-4eb8-bf41-b5bf7b8622d4 service placement] REQUEST_METHOD: `GET` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:08:33.070491 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c478370-ec47-4eb8-bf41-b5bf7b8622d4 service placement] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:08:33.070765 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c478370-ec47-4eb8-bf41-b5bf7b8622d4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:08:33.072670 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-4c478370-ec47-4eb8-bf41-b5bf7b8622d4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:08:33.085251 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c478370-ec47-4eb8-bf41-b5bf7b8622d4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:08:33.086979 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c478370-ec47-4eb8-bf41-b5bf7b8622d4 service placement] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:08:33.098162 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1451/2888] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:08:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:08:36.769363 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca70a70-d5b3-4e12-bbd8-9507f69a8488 service neutron] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:08:36.769363 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-2ca70a70-d5b3-4e12-bbd8-9507f69a8488 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:08:36.780463 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca70a70-d5b3-4e12-bbd8-9507f69a8488 service neutron] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:08:36.785071 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df49ebe-de37-4e12-8757-509522b66618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:08:36.785928 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df49ebe-de37-4e12-8757-509522b66618 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:08:36.786204 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df49ebe-de37-4e12-8757-509522b66618 admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:08:36.786471 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df49ebe-de37-4e12-8757-509522b66618 admin admin] PATH_INFO: `/v3/users/ae2a190cf2ef4777a3838cbe9750afea` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:08:36.787782 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df49ebe-de37-4e12-8757-509522b66618 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae2a190cf2ef4777a3838cbe9750afea)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:08:36.789615 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df49ebe-de37-4e12-8757-509522b66618 admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:08:36.916512 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-5df49ebe-de37-4e12-8757-509522b66618 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae2a190cf2ef4777a3838cbe9750afea'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:08:36.923604 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-5df49ebe-de37-4e12-8757-509522b66618 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae2a190cf2ef4777a3838cbe9750afea'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:08:36.924278 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-5df49ebe-de37-4e12-8757-509522b66618 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae2a190cf2ef4777a3838cbe9750afea'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:08:36.936052 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-5df49ebe-de37-4e12-8757-509522b66618 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae2a190cf2ef4777a3838cbe9750afea'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:08:36.936684 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-5df49ebe-de37-4e12-8757-509522b66618 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae2a190cf2ef4777a3838cbe9750afea'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:08:36.999350 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1438/2889] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 23:08:36 2022] DELETE /identity/v3/users/ae2a190cf2ef4777a3838cbe9750afea => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:08:37.300955 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c478370-ec47-4eb8-bf41-b5bf7b8622d4 service placement] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:08:37.316957 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c478370-ec47-4eb8-bf41-b5bf7b8622d4 service placement] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:08:37.378704 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0acf7e-a6f8-4cbe-a6e7-f555bc796dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:08:37.379626 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0acf7e-a6f8-4cbe-a6e7-f555bc796dc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:08:37.379933 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0acf7e-a6f8-4cbe-a6e7-f555bc796dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:08:37.380209 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0acf7e-a6f8-4cbe-a6e7-f555bc796dc4 admin admin] PATH_INFO: `/v3/projects/24a2b49707184c4093e713f0bbc5b00c` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:08:37.381384 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0acf7e-a6f8-4cbe-a6e7-f555bc796dc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24a2b49707184c4093e713f0bbc5b00c)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:08:37.383358 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0acf7e-a6f8-4cbe-a6e7-f555bc796dc4 admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:08:37.496958 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-6c0acf7e-a6f8-4cbe-a6e7-f555bc796dc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24a2b49707184c4093e713f0bbc5b00c', 'request_id': 'req-6c0acf7e-a6f8-4cbe-a6e7-f555bc796dc4'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:08:37.503219 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-6c0acf7e-a6f8-4cbe-a6e7-f555bc796dc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24a2b49707184c4093e713f0bbc5b00c', 'request_id': 'req-6c0acf7e-a6f8-4cbe-a6e7-f555bc796dc4'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:08:37.564116 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-6c0acf7e-a6f8-4cbe-a6e7-f555bc796dc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24a2b49707184c4093e713f0bbc5b00c', 'request_id': 'req-6c0acf7e-a6f8-4cbe-a6e7-f555bc796dc4'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:08:37.575391 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-6c0acf7e-a6f8-4cbe-a6e7-f555bc796dc4 admin admin] The token cache is being invalidate because project 24a2b49707184c4093e713f0bbc5b00c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:08:37.575833 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-6c0acf7e-a6f8-4cbe-a6e7-f555bc796dc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:08:37.576345 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-6c0acf7e-a6f8-4cbe-a6e7-f555bc796dc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:08:37.577868 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1452/2890] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 23:08:37 2022] DELETE /identity/v3/projects/24a2b49707184c4093e713f0bbc5b00c => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:08:37.581888 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df49ebe-de37-4e12-8757-509522b66618 admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:08:37.583304 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-5df49ebe-de37-4e12-8757-509522b66618 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:08:37.596791 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df49ebe-de37-4e12-8757-509522b66618 admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:08:37.652072 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89626c6-d75d-4e0c-8ef6-f5cac4fb3a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:08:37.652937 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89626c6-d75d-4e0c-8ef6-f5cac4fb3a3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:08:37.653211 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89626c6-d75d-4e0c-8ef6-f5cac4fb3a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:08:37.653481 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89626c6-d75d-4e0c-8ef6-f5cac4fb3a3c admin admin] PATH_INFO: `/v3/users/770c05b9ae99483e977aa3218a088ac1` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:08:37.654649 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89626c6-d75d-4e0c-8ef6-f5cac4fb3a3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=770c05b9ae99483e977aa3218a088ac1)` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:08:37.656560 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89626c6-d75d-4e0c-8ef6-f5cac4fb3a3c admin admin] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:08:37.661634 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:08:37.853856 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-a89626c6-d75d-4e0c-8ef6-f5cac4fb3a3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '770c05b9ae99483e977aa3218a088ac1'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:08:37.860918 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-a89626c6-d75d-4e0c-8ef6-f5cac4fb3a3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '770c05b9ae99483e977aa3218a088ac1'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:08:37.861567 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-a89626c6-d75d-4e0c-8ef6-f5cac4fb3a3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '770c05b9ae99483e977aa3218a088ac1'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:08:37.873709 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-a89626c6-d75d-4e0c-8ef6-f5cac4fb3a3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '770c05b9ae99483e977aa3218a088ac1'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:08:37.874329 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.notifications [None req-a89626c6-d75d-4e0c-8ef6-f5cac4fb3a3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '770c05b9ae99483e977aa3218a088ac1'} {{(pid=87168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:08:37.959368 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1439/2891] 10.222.0.52 () {58 vars in 1223 bytes} [Thu Jul 21 23:08:37 2022] DELETE /identity/v3/users/770c05b9ae99483e977aa3218a088ac1 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:08:38.195617 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0acf7e-a6f8-4cbe-a6e7-f555bc796dc4 admin admin] Authenticating user token {{(pid=87167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:08:38.197615 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.fernet_utils [None req-6c0acf7e-a6f8-4cbe-a6e7-f555bc796dc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:08:38.213106 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0acf7e-a6f8-4cbe-a6e7-f555bc796dc4 admin admin] Validating token access rules against request {{(pid=87167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:08:38.270524 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad5a6c4-928c-4056-8b43-20ef39191e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '031023a71a014731ad7c782eb926f065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd47c52ce443d40eaa4e635852fb964cd', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:08:38.271436 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad5a6c4-928c-4056-8b43-20ef39191e6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:08:38.271726 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad5a6c4-928c-4056-8b43-20ef39191e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:08:38.272008 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad5a6c4-928c-4056-8b43-20ef39191e6b admin admin] PATH_INFO: `/v3/projects/42ade7e26dde4d428b1319ef8e45626a` {{(pid=87167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:08:38.273203 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad5a6c4-928c-4056-8b43-20ef39191e6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=42ade7e26dde4d428b1319ef8e45626a)` {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:08:38.275790 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad5a6c4-928c-4056-8b43-20ef39191e6b admin admin] RBAC: Authorization granted {{(pid=87167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:08:38.296374 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:08:38.369618 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-dad5a6c4-928c-4056-8b43-20ef39191e6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42ade7e26dde4d428b1319ef8e45626a', 'request_id': 'req-dad5a6c4-928c-4056-8b43-20ef39191e6b'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:08:38.375748 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-dad5a6c4-928c-4056-8b43-20ef39191e6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42ade7e26dde4d428b1319ef8e45626a', 'request_id': 'req-dad5a6c4-928c-4056-8b43-20ef39191e6b'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:08:38.473609 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-dad5a6c4-928c-4056-8b43-20ef39191e6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42ade7e26dde4d428b1319ef8e45626a', 'request_id': 'req-dad5a6c4-928c-4056-8b43-20ef39191e6b'} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:08:38.485430 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-dad5a6c4-928c-4056-8b43-20ef39191e6b admin admin] The token cache is being invalidate because project 42ade7e26dde4d428b1319ef8e45626a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 23:08:38.485716 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-dad5a6c4-928c-4056-8b43-20ef39191e6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:08:38.486358 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG keystone.notifications [None req-dad5a6c4-928c-4056-8b43-20ef39191e6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 23:08:38.487931 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: [pid: 87167|app: 0|req: 1453/2892] 10.222.0.52 () {58 vars in 1232 bytes} [Thu Jul 21 23:08:38 2022] DELETE /identity/v3/projects/42ade7e26dde4d428b1319ef8e45626a => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 23:08:47.955780 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:08:48.502333 dv-cinder-850697-1-iscsi devstack@keystone.service[87167]: DEBUG dbcounter [-] [87167] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 23:11:36.044821 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89626c6-d75d-4e0c-8ef6-f5cac4fb3a3c admin admin] Authenticating user token {{(pid=87168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 23:11:36.046219 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-a89626c6-d75d-4e0c-8ef6-f5cac4fb3a3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:11:36.187824 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89626c6-d75d-4e0c-8ef6-f5cac4fb3a3c admin admin] Validating token access rules against request {{(pid=87168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 23:11:36.246791 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bfc502-45b2-4ddc-9d2c-59ef77a9da03 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10c0bed169344a77bfad29624c5e5a8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa11ab8229b642ce962193b74001d3bc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 23:11:36.247463 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bfc502-45b2-4ddc-9d2c-59ef77a9da03 service placement] REQUEST_METHOD: `GET` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 23:11:36.247748 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bfc502-45b2-4ddc-9d2c-59ef77a9da03 service placement] SCRIPT_NAME: `/identity` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 23:11:36.248027 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bfc502-45b2-4ddc-9d2c-59ef77a9da03 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 23:11:36.249801 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.fernet_utils [None req-00bfc502-45b2-4ddc-9d2c-59ef77a9da03 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 23:11:36.261565 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bfc502-45b2-4ddc-9d2c-59ef77a9da03 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 23:11:36.263160 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bfc502-45b2-4ddc-9d2c-59ef77a9da03 service placement] RBAC: Authorization granted {{(pid=87168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 23:11:36.273001 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: [pid: 87168|app: 0|req: 1440/2893] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Jul 21 23:11:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 23:11:46.268793 dv-cinder-850697-1-iscsi devstack@keystone.service[87168]: DEBUG dbcounter [-] [87168] Writing DB stats keystone:SELECT=89 {{(pid=87168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}